The technicalities

obracajacy sie tekst
podwojna strzalka w dol
all articles
Apache Airflow data pipelines devops monitoring sysops
The technicalities min read

Monitoring Airflow jobs with TIG 1: system metrics

Like many server applications, Airflow can – and should – be monitored for metrics and logs. In this article, we will look into the former…

check out.
Apache Airflow DAG dependencies data pipelines job orchestration
The technicalities 5 min read

Databricks – Photon

The Databricks platform offers two execution engines for the clients: the standard Apache Spark (available as an open-source application) and one with Photon enhancement that…

check out.
Apache Airflow DAG dependencies data pipelines job orchestration
The technicalities 6 min read

Airflow — pools and mutexes.

Although the ideal data pipeline is made of idempotent and independent tasks, there are some cases when setting up a mutex (a.k.a. part of the…

check out.
Deployment Strategies for LLMs LLM LLM Benchmarks Understanding LLM Licensing LLM Performance Trade-offs Open Source LLM Deployment
The technicalities 8 min read

What you need to know before deploying Open Source LLM

Navigating the complexities of deploying open-source Large Language Models (LLMs) can be daunting. From understanding licensing restrictions and making crucial decisions about accuracy, speed, and cost trade-offs, to comprehending benchmark evaluations and exploring deployment strategies, this guide provides essential insights for leveraging open-source LLMs effectively in your projects.

check out.
Apache Airflow DAG dependencies data pipelines job orchestration
The technicalities 7 min read

Passing information between DAGs in Airflow.

There are data pipelines where you must pass some values between tasks – not complete datasets, but ~ kilobytes. This can be managed even within…

check out.
Chatbot Development Conversational AI Custom LLM Integration LLM Nvidia NeMo-Guardrails
The technicalities min read

NeMo-Guardrails

Building a dedicated chatbot is both challenging and dangerous. At company X, the model should talk about X’s offer and, ideally, nothing else to save…

check out.