We help you replicate data
to the cloud in real-time
Our Product
Calabash — Build Real-Time Data System in Cloud
Calabash is an online software tool for real-time data ingestion and ETL processing on a cloud platform. Using Calabash, you can collect data from various sources and transform them into desired forms. All in real-time.
Unlike most similar tools on the market, Calabash is not built for developers or experienced engineers. It is for staff with a strong understanding of their business plus light to moderate exposure to the software technologies. But of course, technically well-prepared users are always welcome.
Wonder if Calabash can help you? Ask yourself these questions.
- Do you have a large number of operational data important to your business, but you don’t immediately know how to use or handle them?
- Do you have massive amounts of data coming up too fast every day (hour or minute)?
- Do you have dirty data you want to filter?
- Do you have irregular records you must massage into shape?
If you do, Calabash is your solution.
Calabash’s solution to these challenges is simple: just catch all the data in real-time, then you have more time to figure out what to do with them! You can gradually add real-time ETL processes, known as “data pipelines,” over time as your business evolves
How Does It Work?
Calabash is designed to remove the need for scheduled data ingestion. It launches real-time watchers over your data sources to automatically detect new data in real-time. Once set up, there will be absolutely no human intervention. This real-time watcher-detector-loader is called “reader.”
Readers perform no record processing. They just replicate data verbatim into the cloud. This way, data get loaded into the cloud with a minimum chance for error. The only possible error will be system failures, and in such cases, reload is automatic and started ASAP.
However, it is unlikely you would directly use the raw source data replicated into the cloud platform. You may want to process the data according to your business logic and needs. These will be done by creating real-time processes called “data pipelines.”
Calabash provides a GUI interface for you to design your data pipelines easily. You do not need extensive experience in big-data processing. You just need to be your domain expert, and Calabash will generate implementations for your design.
The pipelines implemented by Calabash are idempotent real-time processes, free of joins of data streams to eliminate process dependencies. Therefore, they are suitable to run on Kubernetes clusters.
Calabash also helps you build cloud infrastructure on which you want to deploy readers and pipelines. You can create complicated Kubernetes cluster, Kafka system, and a security service in the cloud by issuing just one command.
In summary, you design on a high level, let Calabash generate implementation, and deploy the generated implementation to the cloud. Since all generated processes are real-time, you just let them run and watch the result. No further action is needed. You can create sophisticated ingestion plus pipeline processing in about 30 minutes.
Finally, Calabash offers an online repository where you log in to create the design of readers, pipelines, and infrastructure components. You will be able to go back to the drawing board whenever you want. This way, you will never forget what you have done.
Where to Go Next?
- Get started. To get started with Calabash, please read the article about Calabash Architecture and follow the links there.
- Learn about data lake. Setting up real-time readers and pipelines is part of a so-called “data lake” strategy. It is helpful to get a thorough understanding of it. The article What Is Data Lake provides an introduction.
- Learn more about what Calabash can do. You have just seen the tip of an iceberg from the brief introduction on this page. Calabash is a feature-rich tool that can help you in the most complicated areas of your work. These areas include security, web services, Kubernetes, Kafka, plus a lot more. Please read the article on Use Cases of Calabash for details.
- Check documentation. Any time you need to find something, all articles are under the “Doc” menu. You can do keyword searches or browse by article categories.