Getting started with New Relic and Pulumi

Pulumi is an open-source infrastructure-as-code tool that lets you use general-purpose programming languages like TypeScript, Python, Go, and others to build, deploy, and manage cloud infrastructure. With Pulumi, you can manage cloud resources of all kinds, including New Relic dashboards and alerts.

In this guide, you'll learn how to set up New Relic alerts with Pulumi, TypeScript, and Node.js.

Before you begin

To use this guide, you should have some basic knowledge of New Relic, Pulumi, and TypeScript. Additionally, since you'll be working with Pulumi on the command line, you should install the Pulumi CLI, and make sure you've installed a recent version of Node.js as well.

Step 1 of 10

Create a folder for the project

Start by creating a folder for the project. This folder will house two other folders: one for your web application (a small app written with Express.js) and another for the Pulumi program defining the New Relic infrastructure to monitor it.

bash

$mkdir my-project && cd my-project
$mkdir app
$mkdir infrastructure

Step 2 of 10

Create a simple web application

Next, create a small web application and configure it for use with New Relic. In the app folder, install the Express and New Relic packages from npm, then add a placeholder file for the Node.js script to start the application:

bash

$cd app
$npm init -y
$npm install express newrelic --save
$touch index.js

To configure the application to send events to New Relic, copy the newrelic.js configuration file included with the newrelic package into the app folder alongside index.js:

bash

$cp node_modules/newrelic/newrelic.js .

Configure newrelic.js by naming the application my-api and providing your New Relic license key:

// ...
exports.config = {
  app_name: ['my-api'],
  license_key: 'your-ingest-license-key',
  // ...

Finish the app by adding the following code, which defines a single API endpoint, to index.js. Notice that the endpoint returns an HTTP 500: that's on purpose, to help you test the New Relic alert you'll set up next with Pulumi.

// Import the New Relic agent configuration module.
require("newrelic");

const express = require("express");
const app = express();
const port = 3000;

app.get("/", (req, res) => {
    // Deliberately return a server error.
    res.status(500).json({
        message: "Oh no! Something went wrong."
    });
});

app.listen(port, () => {
    console.log(`Now listening on port ${port}...`);
});

Start the application to register it with New Relic. Once it's running locally, you can confirm it's configured by signing in to New Relic and browsing to APM > Services.

bash

$node index.js

With the web application set up, you can move on to creating the New Relic infrastructure to support it.

Step 3 of 10

Create a new Pulumi project

In the infrastructure folder, create a new TypeScript project with the Pulumi CLI. Step through the prompts, accepting the defaults to create a new Pulumi stack, which by default is named dev.

bash

$cd ../infrastructure
$pulumi new typescript

Sign in to Pulumi

If this is your first time running Pulumi, you'll be prompted to sign in to the Pulumi Service, Pulumi's default state-management backend. The Pulumi Service is free for individual use, and self-managed options are also available. See the Pulumi documentation for details.

When the new-project wizard completes, you should be left with the following folder structure:

./my-project
├── app
│   ├── index.js
│   ├── newrelic.js
│   ├── package-lock.json
│   └── package.json
└── infrastructure
    ├── index.ts
    ├── package-lock.json
    ├── package.json
    ├── Pulumi.yaml
    └── tsconfig.json

Step 4 of 10

Install and configure the New Relic provider

Still in the infrastructure folder, install the @pulumi/newrelic package from npm:

bash

$npm install @pulumi/newrelic

Then configure the New Relic provider using Pulumi's configuration system, which has built-in support for encrypted secrets.:

bash

$pulumi config set newrelic:accountId 123456
$pulumi config set newrelic:apiKey NRAK-XXXXXXXXXXXXXXXXXXXXXXXXXXX --secret

Caution

Be sure to use the `--secret` flag when setting your New Relic API key. This will ensure that your API Key is encrypted both in transit and at rest, and that you may safely commit your stack's configuration file to version control.

See Pulumi configuration docs for details.

Step 5 of 10

Define an alert policy and condition

Replace the contents of index.ts with the following code, which imports the @pulumi/pulumi and @pulumi/newrelic libraries and declares a single New Relic AlertPolicy:

import * as pulumi from "@pulumi/pulumi";
import * as newrelic from "@pulumi/newrelic";

const policy = new newrelic.AlertPolicy("alert-policy");

Next, define an NrqlAlertCondition to trigger a critical alert when the application's error rate exceeds a given threshold. Add the following code to index.ts.

A few things to note about this code:

It uses the New Relic provider's getEntity function to look up the APM service you created in Step 2, capturing the result in app as a Pulumi output. Outputs are eventual values that you can think of like JavaScript Promises.
It passes the id property of the AlertPolicy to the alert condition as a Pulumi input, converting its underlying value from a string to the expected number with the help of the apply() method common to all Pulumi outputs.
It creates a dependency relationship between these two resources. By passing the id property of the alert policy (an output) as the policyId of the alert condition (an input), Pulumi knows to create the AlertPolicy resource first.

// ...

// Look up the APM service you just created.
const app = newrelic.getEntityOutput({
    name: "my-api",
});

// Define an alert condition to trigger an alert when the
// service's error rate exceeds 1% over a five-minute period.
const condition = new newrelic.NrqlAlertCondition("alert-condition", {
    description: "Alert when errors exceed threshold.",
    policyId: policy.id.apply(id => parseInt(id)),
    violationTimeLimitSeconds: 1800,
    critical: {
        operator: 'above_or_equals',
        threshold: 1,
        thresholdDuration: 300,
        thresholdOccurrences: 'at_least_once',
    },
    nrql: {
        query: pulumi.interpolate`SELECT COUNT(*) FROM TransactionError WHERE (appName = '${app.name}') AND (error.expected IS FALSE OR error.expected IS NULL)`,
    },
});

Step 6 of 10

Define a notification destination and channel

Next, define how you'll be notified of application errors. The NotificationDestination resource defines the type of notification — here, a message to be sent to your email address — and an accompanying NotificationChannel resource configures the destination as a notification channel.

// ...

// Define a notification destination of your email address.
const destination = new newrelic.NotificationDestination("destination", {
    type: "EMAIL",
    active: true,
    properties: [
        {
            key: "email",
            value: "your-email@example.com",
        },
    ],
});

// Define a notification channel for the email destination.
const channel = new newrelic.NotificationChannel("channel", {
    destinationId: destination.id,
    product: "IINT",
    type: "EMAIL",
    properties: [
        {
            key: "subject",
            value: "{{issueTitle}}",
        },
    ],
});

Step 7 of 10

Define a notification workflow

Finally, define a Workflow resource to route policy violations to the notification channel you defined in the previous step.

// ...

// Define a workflow to route policy violations to the notification channel.
const workflow = new newrelic.Workflow("workflow", {
    issuesFilter: {
        name: app.name,
        type: "FILTER",
        predicates: [{
            attribute: "accumulations.policyName",
            operator: "EXACTLY_MATCHES",
            values: [ policy.name ],
        }],
    },
    destinations: [{
        channelId: channel.id,
    }],
    mutingRulesHandling: "NOTIFY_ALL_ISSUES",
});

Step 8 of 10

Deploy the program

Now it's time to deploy this new infrastructure to New Relic with Pulumi.

In the infrastructure folder, start by running pulumi up:

bash

$pulumi up

When you do so, Pulumi assembles a deployment plan based on the resources and relationships defined in your program, and renders that plan in the form of a deployment preview, which it'll ask you to examine and confirm before proceeding:

bash

Previewing update (dev)
     Type                                       Name                Plan
 +   pulumi:pulumi:Stack                        infrastructure-dev  create
 +   ├─ newrelic:index:AlertPolicy              alert-policy        create
 +   ├─ newrelic:index:NotificationDestination  destination         create
 +   ├─ newrelic:index:NotificationChannel      channel             create
 +   ├─ newrelic:index:NrqlAlertCondition       alert-condition     create
 +   └─ newrelic:index:Workflow                 workflow            create
Resources:
    + 6 to create
Do you want to perform this update?
> yes
  no
  details

Choose details to get a closer look at each resource if you like, and when you're comfortable, choose yes to create the dev stack and complete the deployment:

bash

Updating (dev)
     Type                                       Name                Status
 +   pulumi:pulumi:Stack                        infrastructure-dev  created (1s)
 +   ├─ newrelic:index:AlertPolicy              alert-policy        created (1s)
 +   ├─ newrelic:index:NotificationDestination  destination         created (3s)
 +   ├─ newrelic:index:NrqlAlertCondition       alert-condition     created (2s)
 +   ├─ newrelic:index:NotificationChannel      channel             created (12s)
 +   └─ newrelic:index:Workflow                 workflow            created (1s)
Resources:
    + 6 created
Duration: 22s

When the deployment completes, you should be able to sign in to New Relic and verify that the my-api service is now configured with an alert condition set up to notify you by email when the service exceeds a 1% error rate.

Step 9 of 10

Trigger an alert

Now that the service is configured, you can trigger an alert by invoking the Express application's API enpdpoint.

First, navigate to the app folder and start the application:

bash

$cd ../app
$node index.js
Now listening on port 3000...

Then, in another terminal tab or in your browser of choice, invoke the API endpoint to send a few errors to New Relic:

bash

$curl http://localhost:3000/
{"message":"Oh, no! Something went wrong."}

In a few minutes, you should be notified by email of a new incident.

To resolve the incident, update the Express application to return a 200 response instead of 500, restart the app, make a few more requests, and the incident should be resolved by New Relic automatically.

Step 10 of 10

Destroy the stack

Finally, you can destroy the Pulumi stack to remove all of the New Relic infrastructure you've created up to now.

bash

$pulumi destroy
Previewing destroy (dev)
     Type                                       Name                Plan
 -   pulumi:pulumi:Stack                        infrastructure-dev  delete
 -   ├─ newrelic:index:Workflow                 workflow            delete
 -   ├─ newrelic:index:NotificationChannel      channel             delete
 -   ├─ newrelic:index:NrqlAlertCondition       alert-condition     delete
 -   ├─ newrelic:index:AlertPolicy              alert-policy        delete
 -   └─ newrelic:index:NotificationDestination  destination         delete
Resources:
    - 6 to delete

As before, you'll be prompted to confirm the operation before proceeding. When you're ready, choose yes to continue:

bash

$pulumi destroy
Destroying (dev)
     Type                                       Name                Status
 -   pulumi:pulumi:Stack                        infrastructure-dev  deleted
 -   ├─ newrelic:index:Workflow                 workflow            deleted (1s)
 -   ├─ newrelic:index:NrqlAlertCondition       alert-condition     deleted (0.80s)
 -   ├─ newrelic:index:NotificationChannel      channel             deleted (1s)
 -   ├─ newrelic:index:AlertPolicy              alert-policy        deleted (0.77s)
 -   └─ newrelic:index:NotificationDestination  destination         deleted (0.73s)
Resources:
    - 6 deleted
Duration: 6s

Conclusion

Congratulations! You're now officially practicing observability-as-code with New Relic and Pulumi.

To learn more about what you can do with Pulumi and New Relic and take your application monitoring to the next level, check out the Pulumi New Relic provider documentation.