OVERVIEW

Large Language Models (LLMs) can now use plug-ins to access extra tools. But they often respond slowly when these tools are used. This hurts the user experience. 

In this post, we’ll show a simple trick that speeds up LLM responses. It will help you build more practical and efficient LLM-based solutions.

GOALS

1. Speed up bot answers.

EXAMPLE USE CASE

Let’s say we’re using OpenAI’s ChatGPT via langchain for our holiday rental website where people can talk with chat asking about our website’s current holiday offer or some general geography information. We leave the geography part solely to the LLM, but the one with showing our deals we have to implement on our end. Langchain gives this possibility by using tools. 

Let’s say our chatbot function looks already like this:

def get_chatbot_answer(user_message: str) -> str

and the chat has a tool function that looks like this:

def get_holiday_offers(place: str, month: str) -> str

While they’re very easy to use and applicable in this case, the time spent processing tool responses by LLM can be huge. Huge enough to discourage some users from using our chatbot..

SOLUTION

We can divide our chatbot into 2 chatbots, one that will do the same as original, and the other that will decide if we’re using the first one, or the user just wants to get holiday offers (that way we don’t have to use the first, long one). In the other case, we can use the tool ourselves.

So, we can firstly create the first chatbot, that will return 3 outputs, (we can use structured output for that). It will look like this:

def chatbot_preprocessing(user_message: str) -> (bool, str, str)

and will return boolean if the user only wants to see holiday offers, and 2 next variables are place, and month respectively (we only care about those if the boolean is true). Now we can change a flow of our chat a bit:

def get_chatbot_answer_with_preprocessing(user_message: str) -> str:
	only_show_offers, place, month = chatbot_preprocessing(user_message)
	if only_show_offers:
		return get_holiday_offers(place, month)
	else:
		return get_chatbot_answer(user_message)

This way, if the user wants only offers (which is probably more than half the time), the chatbot answers very quickly, because it avoids processing the result by the LLM.

CONCLUSION

It’s very possible to speed up chat conversation not only by getting faster hardware, but also by using some tricks. That particular trick helps a lot with simple chatbots that are made to help website users to quickly see all the available products and services.

The post Speeding up chatbot with tools appeared first on TantusData.

During their workshop at Big Data Conference Europe, Marcin and Bartek guided participants through the essentials of GPT-powered LLM applications. By the end of the session, attendees had a solid grasp of developing with Chains and powerful reasoning Agents, integrating external APIs and context sources, and building solutions that excel in question-answering over documents—all with a focus on accuracy and safety.

What it covered: 

• Foundations of LLMs: Participants discovered which Large Language Models best suited different needs and how to apply them effectively.
• LangChain Essentials: They gained a solid understanding of the LangChain API and learned how Chains and Agents can drive dynamic, intelligent behavior in applications.
• Context and Integrations: Attendees explored how to integrate external APIs, pass context between components, and leverage vector databases for advanced features.
• Question Answering with Your Own Data: They used embeddings and combined ChatGPT, VectorDb, and LangChain to develop robust question-answering systems.
• Reasoning Agents: Participants examined how Agents can utilize real-time tools such as Google Search or Wolfram Alpha for powerful, on-the-fly problem-solving.
• Accuracy and Safety: They delved into techniques for self-querying, hallucination checks, and output moderation to ensure reliable application performance.
• Tuning and Production: Finally, they got a glimpse into real-world deployment, from optimizing embeddings to managing inference and costs.

Missed the Session? Don’t Worry!

Couldn’t make it to our workshop at Big Data Conference Europe? We’ve got you covered. We can bring the same interactive experience straight to your team.

By the End of This Workshop, You Will:

• Gain a strong understanding of ChatGPT-powered LLM applications.
• Master the LangChain API to build and orchestrate Chains and Agents for complex decision-making.
• Develop real-world applications integrating external APIs and data.
• Build reasoning Agents that tackle dynamic problems in real-time.
• Implement crucial techniques for application safety and accuracy.
• Combine ChatGPT, VectorDb, and LangChain into powerful question-answering systems.
• Level up your AI skills and transform your creative ideas into functional, future-ready solutions.

Hands-On Agenda

1. Introduction
   • Overview of various LLMs – what’s good for your use case?
   • Introduction to LangChain
2. Building with Chains and Agents
   • LangChain API
   • Passing context between components
   • Integrations with external APIs and data sources
   • Comparing Chains vs. Agents
3. Question Answering Over Documents
   • Introduction to embeddings
   • Overview of vector databases
   • Converting documents to vectors (plus common gotchas)
   • Building a simple application with ChatGPT, VectorDb, and LangChain
4. Creating Powerful Reasoning Agents
   • Dynamic decision-making
   • Integrations with tools (e.g., Google Search, Wolfram Alpha)
5. Summary & Best Practices
   • Techniques for improving accuracy and safety: self-querying, hallucination checks, and output moderation
   • LLM in production: key challenges and how to tackle them
   • Potentials for tuning: embeddings, inference optimization, cost management

Ready for a Custom Workshop?

Looking to tailor this session to your organization’s specific needs? We can help. Contact us to explore how we can design a workshop that unleashes the power of generative AI for your projects. Let’s build the future together!

The post LLMs and LangChain – Getting Started Guide  appeared first on TantusData.

§ 1 [Definitions]

The phrases and terms used in the content of these Rules shall have the meanings indicated below:
1. „Organizer” – TantusData Sp. z o.o. based in POLAND, ul.  Alexa Niepodległości 132/136 unit 3, 02-544 Warszawa, under KRS number: 0000930059, NIP: 5213944638, REGON: 520344085, being the owner of the TantusData brand;
2. “Rules” – these Rules of the Contest entitled “TantusData Summer Challenge”, binding for the Organizer and Contestants, regulating the terms and conditions of the Contest, in particular terms of participation in the Contest, rights and obligations of the Organizer, Contest Committee and Contestants in relation to their participation in the Contest;
3. “Contest” – the Contest entitled “TantusData Summer Challenge”, binding for the Organizer and Contestants, regulating the terms and conditions of the Contest, in particular terms of participation in the Contest, rights and obligations of the Organizer, Contest Committee and Contestants in relation to their participation in the Contest;
4. “LinkedIn” – the website under the domain www.linkedin.com , where the Contest is partially held, owned by the Microsoft corporation.
5. “Fanpages” – the profile named “@TantusData” on the LinkedIn website available at: https://www.linkedin owned and administered by the Organizer;
6. “Contestant” – a LinkedIn user who has made a correct and effective entry to the Contest, meeting the eligibility requirements for participation in the Contest as described in § 3 of the Rules;
8. “Contest Committee” – a team consisting of persons selected by the Organizer, which evaluates the entries;

9. „Winner” – a User who has submitted a correct and effective entry to the Contest and whose entry was awarded by the Selection Committee in accordance with the provisions of §4 and §6 of these Terms and Conditions.

§ 2 [General provisions]

1. The Organiser is the founder of the prizes.
2. The Organizer is the administrator of personal data provided by Contestants.
3. Providing personal data is optional, but necessary for the Participant to enter the Competition. Persons who make their data available have the right to access, modify or delete them.
4. These Rules define the conditions of the Contest.
5. The Contest is not created, administered, endorsed or sponsored by the social networking site LinkedIn or by the Microsoft Corporation or by Kaggle with Google LLC. “LinkedIn” is a registered trademark by the LinkedIn Corporation.
6. The competition is carried out on User’s profile and Organizers Fanpage.
7. The Organizer’s Contest Committee supervises the correctness and course of the Contest, i.e. providing information on the Contest and dealing with complaints.
8. The Competition is conducted from month 21st July 2023 till September 16th 2023.

§ 3 [Contestants]

1. Participants in the Contest may only be natural persons who are consumers, have full legal capacity, are LinkedIn users, have an active account on LinkedIn, abide by the rules of LinkedIn service and who have accepted these Rules.
2. Contestant declares, that he/she/they:
a. is a natural person with full legal capacity;
b. is domiciled in the territory of European Union or United Kingdom of Great Britain and Northern Ireland;
c. is familiar with the content of the present Rules and voluntarily enters the Contest;

d. agrees to and accepts the terms of these Rules, including the procedure for claiming prizes, and fully accepts them;

e. undertakes to comply with the provisions of these Rules and the rules of the LinkedIn service;

f. has agreed to the processing of his/her/their personal data for the purposes of participating in the Contest;

g. is a registered user of the LinkedIn social network.

3. The employees and collaborators of the Organizer cannot participate in the Contest.

§ 4 [Prize]

1. the Prizes will be awarded to the first three (3) persons who, in the opinion of the Contest Committee, best complete the Contest Task and are selected by the Contest Committee as Winners.

2. The Organizer will award 3 prizes to the Participants who will be declared Winners (“Prizes”), one award per winner. The Prizes in the Contest are respectively:

a) For first place – an office survival kit containing:  50 EURO Amazon gift card, a notebook, 2 post its, 5 pens, a mug and a tea, a bag, a t-shirt, cookies, an adult colouring book, crayons,

b) For second place – an office survival kit containing:  50 EURO Amazon gift card, a notebook, 2 post its, 5 pens, a mug and a tea, a bag, a t-shirt, cookies, an adult colouring book, crayons,

c) For third place – an office survival kit containing:  50 EURO Amazon gift card, a notebook, 2 post its, 5 pens, a mug and a tea, a bag, a t-shirt, cookies, an adult colouring book, crayons,

3. Prizes will be issued only in the form specified herein, without the possibility of exchanging them for another material prize or for a cash equivalent. 

4. The prizes will be issued to the Winners in a manner communicated to them in further communication.

5. The Winners will be informed about the prize and the conditions of Prize collection in the content of the announcement on Fanpage and through a private message sent to the Winners on LinkedIn within 10 working days after the end of the Contest.

6. By entering the Contest, the Contestant agrees that the Organizer may communicate with him/her in connection with the Contest via the LinkedIn account.

7. in order for the Winner to receive the Prize, the necessary data must be sent to the Organizer within 48 hours of the prize communication being sent to the Winner (to the email provided by the Organizer in the message). The necessary data for the Prize release to take place are:

a) name and surname;

b) residence address (for tax purposes)

c) correspondence address (for the purposes of delivering the Prize, it may be different from the residential address);

d) telephone number;

e) e-mail address.

9. Dispatch of the Prizes will take place without unnecessary delay from the day of the end of the Contest. The prizes shall be sent via post to the address indicated by the Winner. The Organizer shall not be liable for the actions or omissions of the delivery service providers.

10. The Winner may surrender the Prize but shall not be entitled to the cash equivalent or any other prize. In the event of the Prize being forfeited, the Organiser reserves the right to award the Prize to another Entrant or decide not to award a Prize. The Winner may not transfer the right to the Prize to third parties.

11. The Winner’s failure to provide the Organiser with the data referred to in point 7 above or exceeding the permissible time of response, or sending incorrect data, the Winner loses the right to the Prize.

12. The Organizer reserves the right to verify whether the Winners fulfil the conditions stipulated in the Regulations as well as in legal regulations. For this purpose, the Organizer may require the Winner to make specific statements, provide specific data or submit specific documents. Failure to comply with the Regulations or relevant provisions of law, as well as refusal to comply with the above demands, may result in the exclusion of a given person from the Contest, and shall entitle the Organiser to refuse to award the Prize without any claims against the Organiser.

13. All participants who complete the Contest Task appropriately as determined by the Contest Committee, will be awarded with online certificates of successful completion. The certificates will be issued within 14 days of the Contest closing date and will be sent to the participants via the LinkedIn’s message service. The certificates confirm only the successful completion of the Contest Task and the Organiser is not responsible for the verification of the User’s completion process, only of the correct solution submitted.

§ 5 [Rules of the Contest]

1. The Contestant’s task is tocreate a model to detect whether the same author has written two texts using the datasets provided by TantusData within the challenge on the Kaggle platform. The data provided was collected from articles from our blog and two publicly available datasets of news and blog posts. Participants can also use external datasets. The participants are asked to create a smart solution which can take advantage of even a small dataset. The participants must present this in the format specified on Kaggle under the evaluation tab. The participants are also asked to write a description in a Kaggle notebook and make it public after the submission deadline. The participants are required to submit their entries in Kaggle under the submissions tab for the competition, as specified in the overview. This is the complete Summer Challenge task (hereinafter: “Contest Task”). Full instructions and guidelines are available on Kaggle under the challenge section.

2. One Contestant may make multiple submissions, but will be asked to select their best attempt at the Contest Task. The Contestant must work alone.

3. Contestant is forbidden to take any action in connection with participation in the Contest contrary to the law and good manners, and to use data obtained in connection with participation in the Contest for illegal purposes. In particular in the content of an answer to a Competition Task a Contestant cannot include:

a) materials and symbols against the law;

b) materials and symbols violating the rights of third parties or the Organizer, especially

those violating intellectual property rights or the personal rights of third parties or the Organizer;

c) expressions commonly regarded as morally reprehensible or socially inappropriate as well as content violating good morals;

d) obscene or pornographic materials and content;

e) materials and symbols propagating violence or discrimination, inciting racial, religious or ethnic hatred, socially recognised as offensive, vulgar, etc.;

f) content violating the rules of so-called “netiquette”;

g) personal data of other people in an unlawful scope, in particular it is forbidden to send an answer to a contest task using someone else’s name and surname in order to impersonate a particular person;

h) materials, the use of which by the Organiser may hinder or prevent the operation of other programs used by the Organiser, especially materials containing computer viruses.

4. The answers to the Contest task, which do not meet the requirements specified in the Rules, will be excluded from the Contest.

5. In order to ensure the proper organisation and conduct of the Contest, and in particular to assess the accuracy of entries and select the Winners, the Organiser appoints a Contest Committee, which will supervise and arbitrate the Contest.

6. The Contest Committee will award the accurate solutions to the Contest Task, most swiftly and appropriately realising the Contest Task. Submissions are evaluated based on the accuracy score between the predicted label and the true target and a description of a scaled and optimal solution that best fulfils the task with sensible use of resources.

7. From among the submitted Contest Tasks, the Contest Committee will select the Winners, indicating the decisive features of the selection and deciding also on the allocation of places and the distribution of Prizes.

8. The Organizer will also inform about the Winners in a public announcement on the Fanpage.

9. Detailed information about the contest will be available on the Organizer’s website.

§ 6 [Conditions of participation in the Contest]

1. Access to the Contest is free of charge.

2. Necessary conditions for participation in the Contest are:

a. acceptance of the Rules and the correct completion of all tasks described in § 5 Paragraph 1 of the Rules;

b. granting by the Contestant consent to the processing of personal data described in these Regulations by the Organizer (in particular consent described in § 8 of the Rules);

c. transfer of copyrights as referred to in §10 of the Rules.

§ 7 [Organiser’s responsibilities]

1. The Organiser shall not be responsible for the accuracy and truthfulness of the data of the Contestants, including the inability to pass the Prizes, due to reasons attributable to the Participant, in particular if the Participant did not provide a real mailing address, or if the data provided is incomplete or outdated.

2. The Organiser declares that it does not control or monitor the content posted by Participants in terms of reliability and truthfulness, subject to actions related to the removal of violations of the Regulations or generally applicable laws.

3. The Organizer reserves the right to exclude from the Contestants whose actions contravene the law or the Rules, LinkedIn rules, in particular Contestants who:

a) post content that contravenes the applicable law, Rules, LinkedIn rules (in particular, containing offensive content, both in the text and graphic layer);

b) Complete the Contest Task in with help (not on their own);

c) Interfere with the Contest mechanism;

4. Organizer is not responsible for any malfunctions in data communications links, servers, interfaces, browsers, LinkedIn platform, Kaggle platform.

5. The Organizer is not responsible for temporary or permanent blocking of the LinkedIn profiles and pages or any of its mobile applications.

§ 8 [Processing of personal data]

1. Personal data of Contestants, including image, shall be processed by Organizer only for the purpose of performing activities necessary for proper conduct of the Contest.

2. Personal data of the Competition Participants will be stored by the Organiser only for the time necessary to conduct the Competition and award the prizes to the awarded Participants.

3. Personal data will be processed for the following purposes: publication of competition posts and their promotion in:

a. any social media of TantusData brand (LinkedIn; YouTube);

b. on the Organizer’s website in the domain www.tantusdata.com ;

4. Participants have the right to access, correct and delete processed data. Data is provided on a voluntary basis, with registration on the ?? submission form (e.g. Google form/name of the server) network required for participation in the Competition. The Organiser is not responsible for the way LinkedIn (the service used) processes personal data.

5. The Contestant has at any time the right to request the deletion of his personal data and the data of minors from his post. Upon deletion of the data, the Contestant loses the possibility to participate in the Contest.

6. The use of personal data takes place on global communication channels, without territorial restrictions and without time limits (until the removal of data by the Contestants or the cessation of the Organizer’s activities).

§ 9 [Copyright]

1. It is forbidden to infringe in any way the intellectual property rights in the Contest, especially the unauthorized use by the Contestants of works authored by third parties.

2. Contestants in respect of whom the Organiser has received information that they are not the authors of Contest Tasks, or do not have the rights to the answers to the Contest Task, are subject to exclusion from the Contest.
3. In the event that an answer to a Contest Task is a work as defined by the Polish Act on Copyright and Similar Rights, the entirety of the Participant’s economic copyright to the answer to the Contest Task and the right to permit the exercise of subsidiary rights, without any time and territorial restrictions, passes to the Organiser. The Organiser shall be entitled to use and dispose of the work – the contest posts – in the following forms of exploitation:
a) within the scope of recording and multiplying the work in whole or in part – the production of copies of the work using a specific technique, including digital reproduction, printing, reprography, magnetic recording and digital technique;
b) within the scope of dissemination of the work, in its entirety or in part, in a manner other than specified in item a) above – placing on social media, websites, publishing in the Organiser’s promotional and advertising materials, exhibiting, displaying, reproducing, as well as broadcasting and rebroadcasting, as well as making the work available to the public in such a way that everyone can have access to it in a place and time selected by themselves, including via the Internet.

§ 10 [Complaints and notifications of violations]

1. Any complaints regarding the way the Contest is carried out should be submitted by Contestant in writing during the Contest, but not later than within 14 (fourteen) days from the date of issuing the Prizes.
2. A complaint submitted after the deadline shall have no legal effect.
3. A written complaint should contain the name, surname, exact address of the Contestant and
a detailed description and justification of the complaint.
4. The complaint should be sent by registered mail or courier service to the address of the Organiser.
5. Claims will be considered in writing within 30 days from their submittance.

§ 11 [Final provisions]

1. The Regulations shall enter into force on the first day of the competition, i.e. July 21st 2023.
2. Second In matters not covered by these Regulations shall apply the provisions of the Polish Civil Code and other applicable laws.
3. Disputes relating to and arising from the Contest will be resolved by a court of law with jurisdiction over the Organiser’s registered office.
4. Organizer reserves the right to change the rules of the Contest during its duration for important reasons, with the reservation that all possible changes in the Rules will not affect the rights of Contest participants acquired on the basis of the Rules before the date of entry into force of these changes. Information about changes will be posted on the Fanpages.
5. A brief description of the Contest rules can be found in advertising and information materials accompanying the Contest, in particular on the Fanpages. All content included in these materials is for information purposes only. Only the provisions of these Regulations are binding.
6. Contest Regulations are available on Fanpages.

The post The TantusData Summer Challenge: terms and conditions appeared first on TantusData.

Introducing the TantusData Summer Challenge

This summer, we’re not just beating the heat, we’re challenging it. At TantusData, we believe in nurturing the approachable spirit of knowledge-sharing, and to celebrate this, we’re excited to present the TantusData Summer Challenge.

Our challenge is an ode to experts in the making and a refresher for the seasoned professionals. It’s a platform to test your skills and learn, as we will post the optimal solution with instructions later. As a cherry on top, there’s a reward – an ‘Office Survival Kit’, filled with a delightful mix of practical and amusing items. This includes a handy notebook, colourful post-its, essential pens, a sarcastic mug paired with a soothing tea, a stylish t-shirt, a tote, appetising cookies for that quick brain boost, and for moments of relaxation, a funny adult colouring book with crayons. To further enhance your office or workspace, we’re including a 50 EURO Amazon gift card as part of the kit. We have three of these exciting kits ready for our top participants. So buckle up and let your creative juices flow!

At the heart of this challenge lies a model on production topic. We teased this during recent ML conferences and now, it’s time for action. The challenge requires the innovative use of a Siamese Network solution to work with small data sets. Entries will be judged based on the quality and appropriateness of the submission. The finer details are provided below.

And that’s not all – all participants who solve the challenge correctly will be awarded certificates.

But wait! We’ve got a bonus round. A side challenge that promises more thrill and more creativity. Stay tuned to our emails and social media channels, specifically LinkedIn, for the announcement.

A little birdie told us that those who participate in this challenge will have a head start for our upcoming autumn challenge. So, we highly recommend you to keep your work saved.

The how’s and when’s

The challenge instructions: Your task, ‘Authorship Comparison’, is simple but thought-provoking: create a smart solution to determine if two texts share the same author. Rather than developing a massive model, we want to see how well you can optimise a limited dataset. The submissions will be assessed based on the scores and the approaches taken.

All submissions must be through the Kaggle submission system, and if you’re not a member yet, creating a login is easy and free. Check out the detailed instructions and resources on the competition page here.

Deadlines: The Authorship Detection challenge opens on the 21st of July 2023. We strongly recommend participants to commence no later than the 11th of September 2023, to provide sufficient time to develop and fine-tune your model.

Important Update: In response to the feedback from our enthusiastic participants, the submission deadline for the Summer Challenge has been extended. All entries will now be accepted until 16th of September, 2023, 23:59 CEST. This provides additional time for both new participants to join and for existing participants to refine their submissions. We wish everyone the best of luck! No submissions will be accepted after this time, so ensure your entry is submitted promptly.

For beginners and those looking for extra support, we’ve provided tips and resources linked within the challenge on Kaggle. These could be very helpful, so make sure to take a look. But remember, going through these resources might take some time in addition to the time needed for the challenge task itself.

Get started now, and best of luck to you all!

The Fine Print: Here’s the link to all the terms and conditions of the challenge. Yet, the basic rules of participation are also listed on the challenge tabs in Kaggle, which will go live tomorrow. So you can view them there.

Get ready to compete, learn, and win. And remember, the clock is ticking. Keep an eye out for our announcements!

Good luck, challengers!

#TantusDataSummerChallenge

The post The Summer Challenge appeared first on TantusData.

J On The Beach: Where Developers and Data Scientists Meet to learn and unlock the potential of Big Data Technologies

EDUCATE, SHARE, SPREAD, AND LEARN

The technical conference brings together developers, data scientists, and DevOps professionals from around the world to explore the latest trends in big data technologies. This immersive event features workshops, hackathons, and technical talks led by top speakers in the field. Attendees will learn about a range of topics, from data collection and stream processing to machine learning, microservices, artificial intelligence, container systems, and more.

You can read all about this year’s edition here.

On the 10 to 12th of May, fantastic international speakers will join the stage to share their top tips and tricks.

Our own Marcin Szymaniuk will share his best practice takeaways on the 12th of May. You can find more about the topic our expert will present here.

Did we mention that it just happens to be Friday and the conference will conclude with a great party in the evening?

So make sure to book your place and join us in the gorgeous Málaga for invaluable insights and strategies, you won’t learn elsewhere.

The post J On the Beach x TantusData appeared first on TantusData.

Sit back and relax as our experts tell you all there is to know about any search project.

Whether you intend to set up a search engine or improve the one you have, there are a few matters, which are crucial to consider before you start.

Benefits of search engines

As you surely know many websites and e-commerce sites benefit greatly from having search engines operating well. But how do they do that? Why do others fail to deliver such ROI?

Are there other needs than simply letting consumers find the products they are looking for faster?

What to prepare before you start to save money and improve ROI faster.

And, does the project really needs to be that expensive and long-term, or is there an alternate option.

Sounds interesting? Make sure to check out our video prepared by our top search experts Maryna and Jakub.

Take me to the video!

The post All about search appeared first on TantusData.

Clutch has recently brought to our attention that we are listed as a leading service provider on Clutch; naming us as one of Poland-headquartered industry game-changer big data analytics firms.

For those few who haven’t heard of Clutch yet, it is a B2B ratings and reviews platform based in Washington, DC. The company evaluates technology service and solutions companies based on the quality of work, thought leadership, and other key highlights from clients’ reviews.

We are a data company specialising in big data solutions. Subsequently, we excel at data platforms, data-driven systems, analytics, and advanced machine learning. As such, our services range from data infrastructure and cloud optimisation to ELT and re-engineering mission-critical systems. You can see a selection of our case studies at Clutch in the portfolio section.

Let’s hear it from the important ones

Our clients’ honest and thorough reviews on Clutch have a tremendous impact on us. Because, their feedback helps us improve our services and establish our name in the industry. They also show that we do deliver in a manner that fits our values. Today, we will share some snippets of the reviews we have received so far.

“We chose TantusData because of how flexible they were in the way of delivery. It was important for us to have their expert working alongside our internal team full time.” – Principal Engineer, UK’s fastest growing online accommodation marketplaces.

This doesn’t come as a surprise. After all, flexibility is at our core. Indeed, we start by focusing on you, to meet where and when you need it, and bring a solution that best fits your company and situation. Therefore, we guarantee the results: what we promise, we deliver. Finally, from start to finish, your project is only taken care of by experts only. Those who have long standing experience in the industry.

“The onboarding and collaboration was smooth since both technical and communication skills of the team were very good”. – Director of Data & Advanced Analytics, large scale retail company

We speak your language and ensure that the problem is not only solved but also in a way and at a pace convenient to you. Whether you have a long-term project, need extra hands on board mid-delivery, or need some crucial last-minute help — we’re quick to respond. 

“Effective communication and very quick turnaround when unexpected problems arise.” – CTO, 360 Restaurant Management System provider

“One of the key resources of the company is their unique skill set. Such a mix of data science and engineering coupled with clear understanding of rules of GDPR and ability to tune performance is hard to find on the market.” – Head of Data Science, one of the largest telecom companies in Europe

We are a flexible, agile, and unequaled group of experts. This means we can address any situation and provide the best solutions, 100% tailored and effective long-term. So, it’s not about the new and popular solutions but the ones that will serve you best.

“I appreciated that they investigated several solutions and delivered them to us with an explanation. They genuinely cared about the outcomes of the models they built and stayed actively involved throughout split testing on live environments.” – Principal Engineer, UK’s fastest growing online accommodation marketplaces.

Simultaneously, TantusData ensures you have all the knowledge you need straight from our specialized team so that you can fully leverage the solution to create value but also confidently make decisions regarding data.
“We could see that they really cared about meeting our needs, but most importantly about the project long term, that it goes well.” – CTA, Depict.ai

Our badges

The whole team at TantusData is proud to be the recipients of the special Clutch Badges for the top providers. To be sure, we will continue to go the extra mile, no matter the project or business size. We are delighted to see our clients appreciate our hard work. If you’re curious to see more about the reviews, check out our Clutch profile. There you can also view selected projects from our portfolio.

We help our clients get up to speed with taking advantage of their data. Contact us now!

The post TantusData, the game changer by Clutch appeared first on TantusData.

Expense optimization is often the main reason for migrating from on-premise to the cloud. The combination of pay-as-you-go and flexible provisioning reduces the problem of overestimated and overprovisioned compute resources. However, in order to actually reduce infrastructure bills, one has to fully understand the cloud pricing model. Otherwise, invoice total may be a huge surprise.

The AWS cost is driven by four metrics:

• compute power & memory,
• storage,
• processing time,
• data transfer out.

However, those are just general guidelines. Using AWS cost calculator and reading through pricing tables should not be omitted. All three scenarios below are real-world examples that happened when using the cloud without estimating the costs.

S3 endpoint – public or private?

PC gives one ability to choose between deploying into a public or private subnet. In case of the latter one, VMs get access to the internet via NAT Gateway, which is charged by hour and data transfer.

Having said that, let’s estimate how much would it cost to send 1PB of data from S3 to a private subnet? If one does not use S3 endpoint and go with VM –> NAT Gateway –> public S3 endpoint path, then the NAT GW charge for data transfer will apply: 0.045 x 1024 x 1024 ~= $47k. The alternative, which is using a private endpoint, is free.

For further reading click HERE

CloudWatch – paid per metric, not point.

Even though CloudWatch is one of the most essential services, it usually does not significantly contribute to the AWS bill. Metrics are charged per number of metrics and data retrieval API calls. Even if one has hundreds of VMs with tens of monitored metrics, the cost will be hardly visible among other expenses.

However, it also means that it may not be suitable for thousands of sparse metrics. Let’s estimate again: 10k time series, each one producing 256 bytes per minute or ~153MB per hour (10 000 x 0.000256 x 60). This would cost $3000 per month (10k x $0.3) for CloudWatch metrics only. For very basic comparison: one m4.large instance with 200G of storage costs $93 per month.

A word of advice

Both of the above situations resulted in a visible spike in the cloud spend. Luckly, in both situations there was budget monitoring and cost reporting already set up. This allows to react quickly, investigate the bill and identify troublesome services.

This example shows that a fully-cloud solution is just a little different than working on an on-premise one. However, estimating expected cost of the whole application, paired with monitoring cloud spent (day-to-day usage) greatly helps to keep expenses reasonable. 

The post How to waste money in the cloud appeared first on TantusData.

Why use salt in repartition? In the previous blog entry we saw how a skew in a processed dataset is affecting performance of Spark jobs. We resolved the problem by repartitioning the dataset by a column which naturally splits the data into reasonably sized chunks. But what if we don’t have such columns in our dataset? Or what if you would like to control a number of files/file size produced by your Spark job? For the sake of this exercise let’s assume that we have 1.1T of data: 7 days of events. This means roughly 165G per day.

Keep in mind that hour attribute we used before splits one day of data into 24 files. Quick back of the envelope calculation leads to conclusion that the size of the file we expect is 165/24=7G. If you are unlucky two different hours might end up in the same file which means 14G file. This is problematic because it might require you to have beefy executors. Other than that it limits the parallelism – the maximum executors you can effectively throw at a problem is 24×7=156 – it would be a problem if your logic required heavy calculations. Because of that using hour column and creating 24 files per day is not the number you aim for in this case.

What can you do if you want to produce more files per output dir? But still control the number…

The trick is quite simple – add a column with random value (salt). The value will be random but you can control the range of generated values. Then you can just use that column when repartitioning. The range you choose will reflect number of files generated for each directory*. After repartitioning the data will be organised into spark partitions in an expected way. As the last step you need to remove the salt column you just introduced – it holds no business value. This is what the code would look like:

df
      .withColumn("year", year(col("eventTimestamp")))
      .withColumn("month", month(col("eventTimestamp")))
      .withColumn("day", dayofmonth(col("eventTimestamp")))
      .withColumn("salt", toInt(rand() * 540))
      .repartition(col("year"), col("month"), col("day"), col("salt"))
      .drop("salt")
      .write
      .partitionBy("year", "month", "day")
      .save(output)

I want to stress on this part:

.withColumn("salt", toInt(rand() * 540))
      .repartition(col("year"), col("month"), col("day"), col("salt"))
      .drop("salt")

… which is generating new ‘salt‘ column with random int from a range of [0, 540], then repartitioning using the salt and eventually removing the column. The salt is only needed in order to distribute your data into 540 separate partitions instead of having single partitioning processing entire day (which is ~165 GB in our case).

Salting prevents from creating a single file per partition which could be too large and could lead to OOM and other errors. By salting you get much better control of number of files (so size of them) per output directory. When you run the job and observe SparkUI:

You can observe that most of the tasks are processing few hundreds megs. You don’t have very heavy tasks and you don’t have many tasks which are doing nothing – exactly what you want to achieve. Other than that you can confirm the above observation by looking into the files written. There are hundreds of them and each of them is few hundreds megabytes in size.

303.4 M  910.2 M  /user/marcin/result/year=2018/month=3/day=1/part-00013-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.4 M  910.1 M  /user/marcin/result/year=2018/month=3/day=1/part-00019-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.6 M  910.8 M  /user/marcin/result/year=2018/month=3/day=1/part-00023-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
605.6 M  1.8 G    /user/marcin/result/year=2018/month=3/day=1/part-00031-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
606.2 M  1.8 G    /user/marcin/result/year=2018/month=3/day=1/part-00043-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.4 M  910.1 M  /user/marcin/result/year=2018/month=3/day=1/part-00044-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.6 M  910.9 M  /user/marcin/result/year=2018/month=3/day=1/part-00049-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.2 M  909.6 M  /user/marcin/result/year=2018/month=3/day=1/part-00055-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.7 M  911.0 M  /user/marcin/result/year=2018/month=3/day=1/part-00062-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.9 M  911.7 M  /user/marcin/result/year=2018/month=3/day=1/part-00066-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.7 M  911.2 M  /user/marcin/result/year=2018/month=3/day=1/part-00071-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.5 M  910.5 M  /user/marcin/result/year=2018/month=3/day=1/part-00073-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.2 M  909.7 M  /user/marcin/result/year=2018/month=3/day=1/part-00078-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.4 M  910.1 M  /user/marcin/result/year=2018/month=3/day=1/part-00079-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.7 M  911.2 M  /user/marcin/result/year=2018/month=3/day=1/part-00082-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.6 M  910.9 M  /user/marcin/result/year=2018/month=3/day=1/part-00083-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.6 M  910.7 M  /user/marcin/result/year=2018/month=3/day=1/part-00085-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.8 M  911.4 M  /user/marcin/result/year=2018/month=3/day=1/part-00086-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.7 M  911.1 M  /user/marcin/result/year=2018/month=3/day=1/part-00090-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet
303.3 M  910.0 M  /user/marcin/result/year=2018/month=3/day=1/part-00096-bcbb5461-1df0-46a1-abe9-c11382f6a94c.snappy.parquet

To conclude:

Firstly, Spark is keeping repartitioned data in the same task is values of the columns used for repartitioning are the same. Secondly, if the amount of data in single task is too large for your use case you can repartition by more columns in repartition clause. Lastly, you can use salt for artificially splitting data into more tasks and keep better control over size of the data processed per task.

The post Spark shuffle – Case #3 – using salt in repartition appeared first on TantusData.

In the previous blog entry we reviewed a Spark scenario where calling the partitionBy method resulted in each task creating as many files as you had days of events in your dataset (which was too much and caused problems). We fixed that by calling the repartition method.

But will repartitioning your dataset always be enough? And is repartitioning always a good idea? Let’s assume you run the code form the previous blog entry (because it worked before, right?):

And let’s say our dataframe (df) is read from HDFS. The data has the following characteristics:

• It’s stored in 2000 blocks, 128 MB each
• It contains events for just for the past week.

So there are 2000*128MB = 256 GB of data, which makes 256/7=36 GB per day. If we put the number on the shuffle diagram it would look like this:

Notice that because you requested spark to repartition the data by day the entire day will end up being processed by a single task. That means there will be tasks processing at least 36GB of data (or more if you are unlucky and one task will process two days). That also means that there will be tasks doing nothing!

Remember

Each spark task is ran inside a JVM. And most likely your executor JVM will be smaller than 36 GB (and let’s ignore size differences that may depend on how data is represented, in memory vs disk, serialization etc.).

Why do we have just a few tasks that are so heavy?

Before, by partitioning the dataset we wanted to ensure that we write just single file per day. And limiting number of files in HDFS is a nice thing to do.  But in this case creating just a single file per day means that a single executor tries hard to write tens of gigabytes of data at the same time. Does that mean it will fail?  It’s likely, but not guaranteed to happen. It all depends on how your executors are configured and the operations you are performing. It’s likely that you will see messages like these:

INFO collection.ExternalSorter: Thread 58 spilling in-memory batch of 6511 B to disk (33 spills so far) 1

What does that mean? It means that in order to avoid memory problems (It can’t keep everything in memory – it’s too large) a Spark task has to spill the data over to disk before actually writing it to HDFS. And the essential problem in this scenario is that it involves extra writing/reading to/from disk. This translates to ‘being slow’ in a Data Engineer’s language.

How to fix the skew?

It is great that we are trying to avoid writing too many small files. But that does not mean that we should try too hard. We don’t need to try writing 36 GB into a single file. Having files larger than 1GB is usually good enough. So how do you make make sure you split the load? One way is to choose an extra column to repartition by – a column like ‘hour‘. It’s cardinality  is very low (1 – 24 hours) and it naturally splits your dataset into 24 buckets. Let’s run a code like this:

df
      .withColumn("year", year(col("eventTimestamp")))
      .withColumn("month", month(col("eventTimestamp")))
      .withColumn("day", dayofmonth(col("eventTimestamp")))
      .repartition(col("year"), col("month"), col("day"), col("hour"))
      .write
      .partitionBy("year", "month", "day")
      .save(output)

When we run it our Spark UI looks like this:

Why is it easier for Spark to rewrite these files?

It’s quite simple. Repartition is an instruction which tells Spark to process the same values together. When we repartition by day we process an entire day in a single task. However,  when we repartition by ‘day‘ and ‘hour‘ we process a single hour of a day in a single task which means there is significantly less data to handle.

How many files will you have? This time you have 7*24=168 files. Each file will be 1.5 GB on average, which is not bad. You probably won’t get the same number of events every hour but let’s not talk about it right now. As you can see, repartitioning can hurt performance if you misuse it. On the other hand, if you use it correctly it can help you achieve a more balanced data distribution between tasks. Moreover, it also allows you to better control the size of the files you write. 

What about using salting?

And you might not like the idea of partitioning your data by ‘hour’ column. Or you may not even have a good column to use in your dataset. What do you do then? The short answer is ‘salting’. The long answer comes in the next blog post Case #3. 

The post Spark shuffle – Case #2 – repartitioning skewed data appeared first on TantusData.