Search engine and true availability.

Case study

Search engine and true availability.

Enhancing search results with real time info integration for the availability status of products.

Challenge

The feature was needed ‘for yesterday’, so it had to be created and deployed fast. Yet, the client didn’t have human resources to do it. There was already a dependency on another team with which we had to create an integration.

Solutions

First step was to create hourly snapshots of availability data. The 2nd step was to create real time data processing.

Technology & Tools

Apache Beam over Dataflow; GCP : PubSub, BigQuery, Monitoring, Logs, Metrics, Cloud Builds, MemoryStore, Storage, Dataflow, Cloud Composer(Airflow); Redis; Python, Java

Client

One of the biggest e-commerce and brick-and-mortar companies and largest furniture retailers.

Opportunity

Consumers expect a swift and frictionless order. This starts from the moment they look at the product range. Every potential client wants the things to arrive ASAP, which means they expect all search results (or at least the top ones) to contain available products. Avoid dissatisfaction. A repetition of search results showing unavailable products without any further information can result in consumer attrition. In this case you lose the lifetime value. Show products you have. For those being backordered, include the expected availability time. This leads to less checkout inconveniences and enables faster decision making, which makes consumers happier. That feeds into the conversion rate and revenue increase.

Delivery

Enabling real time data, which carries the product availability status, to be fed into the search engine results requires solutions that process the big data received in real time. Because the solution was required urgently, the team leveraged the most effective solutions that supported a short deadline. At first the team leveraged GCP components such as dataflow jobs to process the vast amount of messages that come from pub/sub topics. Then they ensured that the quality of data as well as its correctness was and will remain high. This required moving from hourly processing to realtime. For that our experts switched from in-memory to redis storage. This caused the latency to remain low. The two steps are outlined in the graph below. The 1st step included creating the hourly based solution. In the second step the team moved onto the real time data results.