• Log inStart now

Instrument your application with OpenTelemetry

5 min

lab

This procedure is part of a lab that teaches you how to instrument your application with OpenTelemetry.

Each procedure in the lab builds upon the last, so make sure you've completed the last procedure, Set up your lab environment, before starting this one.

Your Python application is running and getting a lot of traffic, so it's time to instrument it. Using OpenTelemetry provides freedom for you to choose your preferred observability platform—New Relic, of course. Also, because it's open source and supported by many actors in the observability space, you can trust its development and authority as a standard, knowing that it will be broadly supported into the future.

Here, you manually instrument your application to generate traces with the OpenTelemetry API. Then, you configure the SDK to send them to New Relic so you can analyze the results later.

Instrument your application

Step 1 of 8

In the terminal window that's running your simulator, press <CTRL-C>.

You should see your simulator shut down. Now you can add some dependencies and update your app logic.

Step 2 of 8

Install the OpenTelemetry SDK and supporting packages into your virtual environment:

bash
$
pip install opentelemetry-api
$
pip install opentelemetry-sdk
$
pip install opentelemetry-exporter-otlp-proto-grpc

Now that you've installed your dependencies, you need to use those dependencies to instrument your application.

Step 3 of 8

Instrumenting your application begins with a tracer provider. A tracer provider is used for holding configurations and for building tracers. Tracers are then used for creating spans. Spans collect information about an operation or process.

In db.py, create a tracer provider:

1
import logging
2
from opentelemetry import trace
3
from opentelemetry.sdk.resources import Resource
4
from opentelemetry.sdk.trace import TracerProvider
5
6
provider = TracerProvider(
7
resource=Resource.create({"service.name": "speedeedeebee"})
8
)
9
trace.set_tracer_provider(provider)
10
11
class DuplicateKeyError(Exception):
12
pass
13
14
class KeyDoesNotExistError(Exception):
15
pass
16
17
db = {}
18
19
def read(key):
20
"""Read key from the database."""
21
global db
22
23
try:
24
value = db[key]
25
logging.debug("Successful read")
26
return value
27
except KeyError as ke:
28
msg = f"Key `{key}` doesn't exist"
29
logging.debug(msg)
30
raise KeyDoesNotExistError(msg)
31
32
def create(key, value):
33
"""Write key:value to the database."""
34
global db
35
36
if key in db:
37
msg = f"Key `{key}` already exists"
38
logging.debug(msg)
39
raise DuplicateKeyError(msg)
40
41
db[key] = value
42
logging.debug("Successful create")
43
return value
44
45
def update(key, value):
46
"""Update key in the database."""
47
global db
48
49
if key in db:
50
db[key] = value
51
logging.debug("Successful update")
52
return value
53
54
msg = f"Key `{key}` doesn't exist"
55
logging.debug(msg)
56
raise KeyDoesNotExistError(msg)
57
58
def delete(key):
59
"""Delete key from the database."""
60
global db
61
62
if key in db:
63
del db[key]
64
logging.debug("Successful delete")
65
return True
66
67
return False
db.py

Here, you created a tracer provider with a resource. A resource describes a service as a collection of attributes. In this resource, you specified name of your service as "speedeedeebee". You also configured your API to use your new tracer provider.

Step 4 of 8

Add a span processor, which processes span data before exporting it to a telemetry consumer:

1
import logging
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8
9
provider = TracerProvider(
10
resource=Resource.create({"service.name": "speedeedeebee"})
11
)
12
provider.add_span_processor(
13
BatchSpanProcessor(
14
OTLPSpanExporter(compression=Compression.Gzip)
15
)
16
)
17
trace.set_tracer_provider(provider)
18
19
class DuplicateKeyError(Exception):
20
pass
21
22
class KeyDoesNotExistError(Exception):
23
pass
24
25
db = {}
26
27
def read(key):
28
"""Read key from the database."""
29
global db
30
31
try:
32
value = db[key]
33
logging.debug("Successful read")
34
return value
35
except KeyError as ke:
36
msg = f"Key `{key}` doesn't exist"
37
logging.debug(msg)
38
raise KeyDoesNotExistError(msg)
39
40
def create(key, value):
41
"""Write key:value to the database."""
42
global db
43
44
if key in db:
45
msg = f"Key `{key}` already exists"
46
logging.debug(msg)
47
raise DuplicateKeyError(msg)
48
49
db[key] = value
50
logging.debug("Successful create")
51
return value
52
53
def update(key, value):
54
"""Update key in the database."""
55
global db
56
57
if key in db:
58
db[key] = value
59
logging.debug("Successful update")
60
return value
61
62
msg = f"Key `{key}` doesn't exist"
63
logging.debug(msg)
64
raise KeyDoesNotExistError(msg)
65
66
def delete(key):
67
"""Delete key from the database."""
68
global db
69
70
if key in db:
71
del db[key]
72
logging.debug("Successful delete")
73
return True
74
75
return False
db.py

The BatchSpanProcessor you used here batches spans before exporting them. This reduces the number of requests you send to New Relic.

Within this span processor, you also configured a span exporter. The exporter is in charge of serializing and sending spans to the consumer. Here, you used OTLP, OpenTelemetry's exchange protocol, and Gzip compression to efficiently transport your telemetry data to New Relic.

You configure your tracer provider with this processor logically prior to setting your tracer provider in the API.

Step 5 of 8

Create two environment variables that you use to configure your OpenTelemetry pipelines. Don't forget to replace the license key placeholder with your real one:

bash
$
export OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.nr-data.net:4317
$
export OTEL_EXPORTER_OTLP_HEADERS="api-key=<YOUR-LICENSE-KEY>"

The OpenTelemetry Protocol (OTLP) endpoint is the url of our OpenTelemetry receiver. Your service sends data directly to New Relic through this endpoint.

Important

https://otlp.nr-data.net:4317 is our US endpoint. If you're in the EU, use https://otlp.eu01.nr-data.net/ instead.

There are several different types of API keys to choose from in New Relic that each serve a different purpose. To instrument your application with OpenTelemetry, you need a license key.

The span exporter you configured in the last step automatically uses these standard environment variables.

Step 6 of 8

Create a tracer:

1
import logging
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8
9
provider = TracerProvider(
10
resource=Resource.create({"service.name": "speedeedeebee"})
11
)
12
provider.add_span_processor(
13
BatchSpanProcessor(
14
OTLPSpanExporter(compression=Compression.Gzip)
15
)
16
)
17
trace.set_tracer_provider(provider)
18
19
tracer = trace.get_tracer(__name__)
20
21
class DuplicateKeyError(Exception):
22
pass
23
24
class KeyDoesNotExistError(Exception):
25
pass
26
27
db = {}
28
29
def read(key):
30
"""Read key from the database."""
31
global db
32
33
try:
34
value = db[key]
35
logging.debug("Successful read")
36
return value
37
except KeyError as ke:
38
msg = f"Key `{key}` doesn't exist"
39
logging.debug(msg)
40
raise KeyDoesNotExistError(msg)
41
42
def create(key, value):
43
"""Write key:value to the database."""
44
global db
45
46
if key in db:
47
msg = f"Key `{key}` already exists"
48
logging.debug(msg)
49
raise DuplicateKeyError(msg)
50
51
db[key] = value
52
logging.debug("Successful create")
53
return value
54
55
def update(key, value):
56
"""Update key in the database."""
57
global db
58
59
if key in db:
60
db[key] = value
61
logging.debug("Successful update")
62
return value
63
64
msg = f"Key `{key}` doesn't exist"
65
logging.debug(msg)
66
raise KeyDoesNotExistError(msg)
67
68
def delete(key):
69
"""Delete key from the database."""
70
global db
71
72
if key in db:
73
del db[key]
74
logging.debug("Successful delete")
75
return True
76
77
return False
db.py

You use this to create spans.

Step 7 of 8

Wrap the logic in each of your database functions with a span:

1
import logging
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8
9
provider = TracerProvider(
10
resource=Resource.create({"service.name": "speedeedeebee"})
11
)
12
provider.add_span_processor(
13
BatchSpanProcessor(
14
OTLPSpanExporter(compression=Compression.Gzip)
15
)
16
)
17
trace.set_tracer_provider(provider)
18
19
tracer = trace.get_tracer(__name__)
20
21
class DuplicateKeyError(Exception):
22
pass
23
24
class KeyDoesNotExistError(Exception):
25
pass
26
27
db = {}
28
29
def read(key):
30
"""Read key from the database."""
31
global db
32
33
with tracer.start_as_current_span("read", kind=trace.SpanKind.SERVER) as span:
34
try:
35
value = db[key]
36
logging.debug("Successful read")
37
return value
38
except KeyError as ke:
39
msg = f"Key `{key}` doesn't exist"
40
logging.debug(msg)
41
raise KeyDoesNotExistError(msg)
42
43
def create(key, value):
44
"""Write key:value to the database."""
45
global db
46
47
with tracer.start_as_current_span("create", kind=trace.SpanKind.SERVER) as span:
48
if key in db:
49
msg = f"Key `{key}` already exists"
50
logging.debug(msg)
51
raise DuplicateKeyError(msg)
52
53
db[key] = value
54
logging.debug("Successful create")
55
return value
56
57
def update(key, value):
58
"""Update key in the database."""
59
global db
60
61
with tracer.start_as_current_span("update", kind=trace.SpanKind.SERVER) as span:
62
if key in db:
63
db[key] = value
64
logging.debug("Successful update")
65
return value
66
67
msg = f"Key `{key}` doesn't exist"
68
logging.debug(msg)
69
raise KeyDoesNotExistError(msg)
70
71
def delete(key):
72
"""Delete key from the database."""
73
global db
74
75
with tracer.start_as_current_span("delete", kind=trace.SpanKind.SERVER) as span:
76
if key in db:
77
del db[key]
78
logging.debug("Successful delete")
79
return True
80
81
return False
db.py

To capture data about the operations in your database functions, you used the tracer.start_as_current_span() context manager. In it, you specified the name of the span and the kind of span it is. Because it's a database server, you specify trace.SpanKind.SERVER.

The API populates some data about the span for you. You'll see the data it captures when you look at your spans in New Relic.

Step 8 of 8

In the success cases, capture the key that was used for each operation:

1
import logging
2
from grpc import Compression
3
from opentelemetry import trace
4
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
5
from opentelemetry.sdk.resources import Resource
6
from opentelemetry.sdk.trace import TracerProvider
7
from opentelemetry.sdk.trace.export import BatchSpanProcessor
8
9
provider = TracerProvider(
10
resource=Resource.create({"service.name": "speedeedeebee"})
11
)
12
provider.add_span_processor(
13
BatchSpanProcessor(
14
OTLPSpanExporter(compression=Compression.Gzip)
15
)
16
)
17
trace.set_tracer_provider(provider)
18
19
tracer = trace.get_tracer(__name__)
20
21
class DuplicateKeyError(Exception):
22
pass
23
24
class KeyDoesNotExistError(Exception):
25
pass
26
27
db = {}
28
29
def read(key):
30
"""Read key from the database."""
31
global db
32
33
with tracer.start_as_current_span("read", kind=trace.SpanKind.SERVER) as span:
34
try:
35
value = db[key]
36
logging.debug("Successful read")
37
span.set_attribute("key", key)
38
return value
39
except KeyError as ke:
40
msg = f"Key `{key}` doesn't exist"
41
logging.debug(msg)
42
raise KeyDoesNotExistError(msg)
43
44
def create(key, value):
45
"""Write key:value to the database."""
46
global db
47
48
with tracer.start_as_current_span("create", kind=trace.SpanKind.SERVER) as span:
49
if key in db:
50
msg = f"Key `{key}` already exists"
51
logging.debug(msg)
52
raise DuplicateKeyError(msg)
53
54
db[key] = value
55
logging.debug("Successful create")
56
span.set_attribute("key", key)
57
return value
58
59
def update(key, value):
60
"""Update key in the database."""
61
global db
62
63
with tracer.start_as_current_span("update", kind=trace.SpanKind.SERVER) as span:
64
if key in db:
65
db[key] = value
66
logging.debug("Successful update")
67
span.set_attribute("key", key)
68
return value
69
70
msg = f"Key `{key}` doesn't exist"
71
logging.debug(msg)
72
raise KeyDoesNotExistError(msg)
73
74
def delete(key):
75
"""Delete key from the database."""
76
global db
77
78
with tracer.start_as_current_span("delete", kind=trace.SpanKind.SERVER) as span:
79
if key in db:
80
del db[key]
81
logging.debug("Successful delete")
82
span.set_attribute("key", key)
83
return True
84
85
return False
db.py

Here, you use the span you created with the context manager to capture the key used in the operation as an attribute.

Restart your simulator

Now that you've changed the application logic, you need to restart your simulator. Make sure you do this in the same terminal window where you set your environment variables:

bash
$
python simulator.py

You've instrumented your application to send traces to New Relic using OTLP. You've also restarted your simulator. Now, it's time to view your data.

lab

This procedure is part of a lab that teaches you how to instrument your application with OpenTelemetry. Now that you've instrumented your app, view your telemetry data in New Relic.

Copyright © 2022 New Relic Inc.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.