HTTP Streaming in Elixir

One of the Elixir web apps we have at Adjust acts as an API gateway — it receives a request, applies authorization and/or authentication and then passes the request along to an actual requested service. As you probably know, if you’re building microservices, this approach is quite common. In our case, the API gateway also stands before a service that is responsible for generating reports. These reports are usually big json or csv blobs and sometimes they are as big as a few hundred megabytes. Because of this, downloading such a report via API gateway just to send it to a client does not sound like a good idea. Below, you can see what happened to our naïve implementation when a number of clients were trying to download sizeable reports.

screenshot

In this blogpost, I’d like to describe how we’ve implemented transparent streaming of HTTP requests directly to a client.

FastHTTP Client

At adjust we recently tried to replace the Go standard http library with fasthttp. Fasthttp is a low allocation, high performance HTTP library, in synthetic benchmarks the client shows a 10x performance improvement and in real systems the server has been reported to provide a 3x speedup. The service we wanted to improve makes a very large number of HTTP requests and so we were very interested in using the fasthttp client.

In the course of making the switch we encountered a number of difficulties. First the fasthttp library presents a very different interface to the programmer which must be adjusted to. Second there were a number of quirks in the implementation which made progress rather slow.

How We Deploy Elixir Apps

We at adjust recently started to use Elixir. We built a couple of small services using the Phoenix framework which successfully went live. In this blogpost I’d like to talk about, I’d say, the most undiscussed topic when it comes to Elixir — deployment.

Dive Into Deeplinking

In the mobile world, deeplinking is a technology that launches an app and opens a specific page once a user clicks a URL on a web page or in another app. We will dive into the details of implementing deeplinking for your app in this article.

Istore: PostgreSQL Documents for Analytical Workloads

Inspired by the PostgreSQL key/value data-type hstore, we developed the istore extension with support for operators like + and aggregates like SUM for semi-structured integer-based data.

While the hstore allows arbitrary textual-data as its keys and values, in an istore document both keys and values are represented and stored as integers. Therefore istore fits nicely in an analytical workload. User journeys, cohort or funnel data, distributional data and many other scenarios can be efficiently modeled and stored in PostgreSQL using istore.

The extension comes with two data types: istore and bigistore, the former having int and the latter bigint as values; keys being int for both. This article demonstrates the efficiency of istore and some of its applications through two examples - aggregating logs and analyzing event funnels.

Writing Postgres Extensions - the Basics

Postgres has a ton of features and offers a wide range of data types, functions, operators, and aggregates. But sometimes it’s just not enough for your use case. Luckily, it’s easy to extend Postgres’ functionality through extension. So why not write your own?