I put this Docker-Compose recipe together to make kicking the tires on Moose—our open-source data-backend framework—almost friction-less.
What you get:
• A single docker compose up that spins up ClickHouse, Redpanda, Redis and Temporal with health-checks & log-rotation already wired.
• Runs comfortably on an 8 GB / 4-core VPS; scale-out pointers are in the doc if you outgrow single-node.
• No root Docker needed; the stack follows the hardening tips ClickHouse & Temporal recommend.
Why bother?
Moose lets you model data pipelines in TypeScript/Python and auto-provisions the OLAP tables, streams and APIs—cuts a lot of boilerplate. Happy to trade notes on the approach or hear where the defaults feel off.
I have a small open-source project, that uses docker compose behind the scenes, to help startup any service. You can look to add it in (or I am also happy to add it in) and then users are one command away from running it (insta moose). Recently just added in lakekeeper and various data annotation tools.
Interesting. How do you do dependencies between those pieces of infrastructure if there's any? For example, in our Docker Compose file, we have temporal that depends on progress and then moose depends on temporal. How is that expressed in Insta-Infra?
pitah1 11 hours ago [-]
It leverages docker compose 'depends_on' for the dependencies (https://docs.docker.com/compose/how-tos/startup-order/). For example, airflow depends on airflow-init container to be completed successfully which then depends on postgres.
How Moose compares to more traditional ELT data pipeline orchestration frameworks, like Airflow, Dagster, dbt, DuckDB for transformation steps.
I think one of the reasons to use an orchestration framework is integations.
mitchellsuzuki 13 hours ago [-]
this is too perfect.
as an SRE who often needs to hand roll my own deployments in k8s or w/e medium, these are the docs that really accelerate my path to production.
faeeafeae 13 hours ago [-]
[dead]
Rendered at 12:47:18 GMT+0000 (UTC) with Wasmer Edge.
What you get:
• A single docker compose up that spins up ClickHouse, Redpanda, Redis and Temporal with health-checks & log-rotation already wired.
• Runs comfortably on an 8 GB / 4-core VPS; scale-out pointers are in the doc if you outgrow single-node.
• No root Docker needed; the stack follows the hardening tips ClickHouse & Temporal recommend.
Why bother?
Moose lets you model data pipelines in TypeScript/Python and auto-provisions the OLAP tables, streams and APIs—cuts a lot of boilerplate. Happy to trade notes on the approach or hear where the defaults feel off.
Docs: https://docs.fiveonefour.com/moose/deploying/self-hosting/de...
18-min walkthrough video: https://www.youtube.com/watch?v=bAKYSrLt8vo
insta-infra: https://github.com/data-catering/insta-infra
https://github.com/data-catering/insta-infra/blob/main/cmd/i...
https://mooseframework.inl.gov/
I think one of the reasons to use an orchestration framework is integations.