Case study

Introducing Databricks for a fintech startup

Migration from MS SQL to Databricks


A finance company has to migrate its analytics to a big data solution, as its to-date approach becomes very cost-inefficient. The previous solution was also becoming a bottleneck as onboarding significantly bigger clients was impossible. The IT team is very proficient in both application development and cloud administration but is new to the Apache Spark ecosystem, so needed proper onboarding.


The TantusData team worked closely with the client’s team to discover and implement the analytics algorithms. There was a very short loop of development - testing - validation cycle to identify any differences between the expected and actual results early on and correct them. The client’s team was also up to date with the created data jobs to get familiar with it and be able to take it over and extend it if required.

Technology & Tools

Apache Spark
MS SQL (migration data source)


Tradeteq is a fintech startup based in the UK.


The Databricks platform is more cost-efficient and offers more flexible scaling options than an SQL database (including cloud offerings). Keeping the SQL solution would ramp up the infrastructure cost by an order of magnitude, while the Databricks & Spark platform scales linearly and is significantly more cost-efficient in our client’s case.


As a startup, the company focuses on short delivery time, quick feedback loop, and transparency regarding the current state of features in progress. TantusData has successfully accommodated this approach and delivered accordingly. Within a month and a half, all major data jobs were migrated from Azure Functions applications to Apache Spark jobs. The process includes onboarding and familiarising the client’s engineers with the solution.


The implemented data platform resolved the challenge of scaling the existing MSSQL-backed batch processing. The platform is ready for a 100x increase in events without affecting the transactional backend system. It not only enables future growth but is significantly cheaper than the previous solution.

More case studies

Case study

Introducing Apache Airflow

Transforming Telecom Data Pipelines for Enhanced Efficiency and Scalability

learn more.
Case study

Customising Databricks for Comprehensive Business Metrics and Data Integration

Integrating sales data for business analysis using Apache Spark

learn more.