Enabled Faster Data Retrieval by Developing an Elasticsearch Powered Application

The Customer and Their Product’s Scope
Our customer is a global transportation solutions provider with a prominent presence in Europe and the USA. Primarily, they provide end-to-end solutions for the public transportation and mobility domains which include automatic fare collection systems and intermodal transport control systems. Both systems have support provided for back-office and front-office operations.

Their Automatic Fare Collection Back-office (AFC BO) system is used in physical locations, web portals and automatic vending machine to sell personalized travel subscriptions or anonymous travel subscriptions.

The Customer’s Critical Requirements
The customer wanted to ensure the presence of the following in their AFC BO system:

• Delivering superior customer service by providing quick and relevant search results for their preferred services. This required the pooling of scattered data across multiple tables from an enormous product database
• Enabling centralized operational logging

Our Solution’s Approach
Any relational database works comparatively slower while handling huge data sets. This slows down the fetching of search results through queries from the database. An optimized relational database management system (RDMS) could solve this issue, but that also comes with its own set of limitations. Not every field can be indexed, and updating all rows to heavily indexed tables is a lengthy and painful process. Businesses therefore, look for alternate data storage methods for ensuring quick retrieval of necessary details. This can be achieved by adopting NoSQL instead of a RDBMS for storing necessary data.

Our proposed solution integrated Elasticsearch to the existing system as a secondary storage. This was done by storing denormalized data in a single document to quicken the data retrieval process. The elastic data was in constant sync with normalized RDMS data. We also provided a centralized logging system by adopting an ELK (Elasticsearch, Logstash, and Kibana) stack.

Let us delve into Elasticsearch as an open source tool, to see how its implementation helped meet the customer’s requirements.

Understanding Elasticsearch
Elasticsearch is an open source search and analytics engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with a HTTP web interface and schema-free JSON documents. It is developed in Java and supports clients in multiple languages. As a document-oriented search engine, it is designed to store, retrieve, and manage data, be it document-oriented, structured, unstructured, or semi-structured in nature.

Elasticsearch has its own Query Domain Specific Language, wherein you specify the query in JSON format. The query DSL is powerful and designed to handle real world query complexity through a single query.

Elasticsearch Use Cases
Some of the use cases for Elasticsearch are as follows:

• It is primarily used for text-based search, wherein textual content is voluminous data is searched for the best match against a specific term or phrase
• For delivering faster product search results in e-commerce businesses
• Facilitating geo-localized product search results by providing geo-search capabilities
• Processing behemoth daily operational events to analyze logs ensuring consistent system performance or anomaly detection

Benefits of Elasticsearch
Compared to most NoSQL databases, Elasticsearch is much more focused on the search functionality. It comes equipped with a rich and powerful HTTP RESTful API that enables search results in near real-time.

Here are some of the key benefits in using Elasticsearch:

• Provides faster access to data
• Schema-free experience, unless mapping is provided
• Effortlessly manages huge amount of data
• Effective scalability of the search engine due to its Distributed Architecture Design
• A well-documented API which makes its integration an easy task
• Is also easily available as Docker images
• Provides operational flexibility for both on-premise and Cloud-based plans

The End Result of This Solution
The customer’s AFC BO system received the following benefits after implementing our solution:

• Faster data retrieval which improved the searching experience for customers
• Some of the business validations were done with existing data so that system does it on elastic data itself which improved the travel subscription sale service.
• Centralized logging through Logstash with visualization through Kibana ensured consistency in system performance and anomaly detection, thereby improving customers’ experience
• Notifying business partners faster of bulk travel subscription renewals by their beneficiaries
• Faster retrieval of transactional data through RESTful API which enabled their business analytics capabilities

Sasken has been at the forefront of the Digital revolution, having provided multiple unique solutions to its customers and their diverse needs. Find out more about our Digital expertise here.