To gather APM events (called transactions and spans), errors and metrics, the Python agent instruments your application in a few different ways. These events, are then sent to the APM Server. The APM Server converts them to a format suitable for Elasticsearch, and sends them to an Elasticsearch cluster. You can then use the APM app in Kibana to gain insight into latency issues and error culprits within your application.

Broadly, we differentiate between three different approaches to collect the necessary data: framework integration, instrumentation, and background collection.

Framework integrationedit

To collect data about incoming requests and background tasks, we integrate with frameworks like Django, Flask and Celery. Whenever possible, framework integrations make use of hooks and signals provided by the framework. Examples of this are:

Framework integrations require some limited code changes in your app. E.g. for Django, you need to add elasticapm.contrib.django to INSTALLED_APPS.

What if you are not using a frameworkedit

If you’re not using a supported framework, for example, a simple Python script, you can still leverage the agent’s automatic instrumentation. Check out our docs on instrumenting custom code.

Instrumentationedit

To collect data from database drivers, HTTP libraries etc., we instrument certain functions and methods in these libraries. Our instrumentation wraps these callables and collects additional data, like

We use a 3rd party library, wrapt, to wrap the callables. You can read more on how wrapt works in Graham Dumpleton’s excellent series of blog posts.

Instrumentations are set up automatically and do not require any code changes. See Automatic Instrumentation to learn more about which libraries we support.

Background collectionedit

In addition to APM and error data, the Python agent also collects system and application metrics in regular intervals. This collection happens in a background thread that is started by the agent.

In addition to the metrics collection background thread, the agent starts two additional threads per process:

Note that every process that instantiates the agent will have these three threads. This means that when you e.g. use gunicorn or uwsgi workers, each worker will have three threads started by the Python agent.