A Telecom Service Provider Handles Huge Volumes of Data Using PostgreSQL

PostgreSQL time!

A telecom service provider searching for a scalable database solution to handle a humongous database, found its answer in PostgreSQL. Siddhartha Singh, one of the EnterpriseDB PostgreSQL contest winners, shares with the LINUX For You team how he created a database for NextGen.

It’s official — open-minded programmers have a thing for open source. Siddhartha Singh is a hands-on programmer who plays the role of an IT architect at Reuters. He has been working on open source for 10 years now, and has observed that people tend to consider open source only when they don’t find what they want in proprietary solutions — but programmers, on the other hand, tilt towards it after considering the licensing terms. “What’s interesting is that open source allows you to play around with already available code, and you can even contribute to the code,” Singh endorses.

Singh has had varied experiences having worked on Android application development, MySQL database, C++, and PHP/Apache/PostgreSQL. He embraced PostgreSQL during his stint with Mavenir Systems, and was so impressed with it that he started using it extensively in other projects. The first project Singh used it on was for NextGen, a telecom services group.

As a telecom service provider, NextGen stores the subscriber information of about one million customers of telecom operators in Japan. It required a database to handle centralised subscriber information, including details like names, addresses, credit balances, data on when users are on roaming tariffs, etc. The data gets incorporated into the operator’s network from NextGen’s database.

The organisation previously used MySQL to store this database, and Singh already knew that PostgreSQL is another alternative to it. He reasons, “Back then, MySQL was more popular, as the contribution to MySQL was much faster and more regular, but after its acquisition by Oracle, there has been apprehension in considering MySQL as an open source solution. Moreover, the support charge is also high in comparison to PostgreSQL.”

Using PostgreSQL as a database solution came with a minimal cost of about Rs 18 lakh for an unlimited plan that allows the company to deploy as many PostgreSQL servers as required in the organisation. NextGen also purchased the support license from Mavenir, and gets support for the code that they provided.

NextGen’s IT team relies on PostgreSQL for database support, if the need arises. Singh had considered other solutions before deploying PostgreSQL. “Firebird and VoltDB were some of our options, but they had some limitations: VoltDB works only on 64-bit machines, and it is a memory-based solution, i.e., if a machine shuts down, you will end up losing data. Firebird, on the other hand, is not rich in features like scalability, high availability etc, and Oracle would have proved 10 times more expensive than open source,” he reveals.

Meanwhile, PostgreSQL has existed for a while, and it is now considered mature, as it comes with its own support system. High availability, in-built replication, and scalability were the major parameters the company was looking for — and PostgreSQL provided them all. Singh decided to go for PostgreSQL as he felt it was most suitable for NextGen’s needs.

Feature-rich database

PostgreSQL has proven itself as a sustainable solution, as it has not crashed once in its six months of deployment, considering that it is being used in a mission-critical project. The system hosts a good volume of data and information (about one million subscribers) on a dual-core machine. NextGen has a 16-core server system. PostgreSQL’s built-in concurrency utilises all the CPUs and memory, and if there are multiple requests from clients to view data, the need is easily met.

Singh feels that using PostgreSQL 9.x at NextGen offers good data safety, as it provides asynchronous stream-based replication. Singh explains, “It is necessary to have a backup of the database, considering that this is a mission-critical project, and information is critical to NextGen.”

Replication used to need a system called SLONY (an automated process that replicates data in real-time from the live server to the backup/stand-by server) in earlier versions of the database. With 9.x, it is built into PostgreSQL, considerably simplifying backup to the stand-by server.

PostgreSQL also provides other useful features like schemas, which help to handle mission-critical applications in that they give you the option to limit people’s access to the database, keeping data secure.

A carefree migration

The transition from MySQL to PostgreSQL was a smooth one, as the new solution comes with tools that ease the migration. PostgreSQL also runs on almost every platform. “The hardware you use can be as small as a PC, and it will still give you great results. It is fast enough to perform 2,000 requests per second, and in this case, with a stand-alone server,” Singh asserts. The migration did not require any additional investment, as the required equipment was already available.

NextGen’s database works on two servers — one active, and one stand-by. A disk space of 3 GB, and 4 GB of RAM with a quad-core CPU are being used to store and handle information about one million subscribers.

In the future, Singh intends to consider the PostgreSQL database instead of Firebird at Reuters. “We are evaluating the features needed, and we’ll take a decision accordingly,” he concludes.

All published articles are released under Creative Commons Attribution-NonCommercial 3.0 Unported License, unless otherwise noted.
Open Source For You is powered by WordPress, which gladly sits on top of a CentOS-based LEMP stack.

Creative Commons License.