Using Webhooks

Why use Webhooks?

Webhooks are a powerful resource that you can use to automate your workflow and improve the scalability of your implementation.

With the exception of the step where you Create a Business via the API, the Middesk workflow is largely asynchronous.

Since Business entities are not static, it is common for Middesk to find updated information about that Business as it continues to monitor it. If you have a Webhook set up, Middesk can immediately notify you of that change upon discovery.

Some examples of when Middesk would send a Webhook are:

• Approving a new account when a Business is complete and meets requirements
• Notifying a team when a Business requires further review
• Updating account profiles with accurate, validated business information

Creating a Webhook

If you provide Middesk a Webhook URL, we will send requests to it to notify you any time an event takes place for a Business.

📘

Registering Webhooks

To register your Webhook URL with Middesk, use our Webhooks API or go to the Webhooks section in your Dashboard Settings.

Handling Webhook Events

Setup an HTTPS endpoint function in your application that can accept webhook requests with a POST method.

At a high-level, your endpoint function should perform the following functions:

1. Handle POST requests with a JSON payload consisting of an Event Object.
2. Quickly responds with a successful status code (2xx) prior to completing any time-intensive processing that might yield a request timeout.

Example Endpoint

require 'json'

# Sinatra
post '/my/webhook/url' do
  payload = request.body.read
  event = JSON.parse(payload)
  
  case event.type
  when 'business.created'
    business = event.data.object
    puts 'Business created!'
  when 'business.updated'
    business = event.data.object
    puts 'Business updated!'
  # ... handle other event types
  else
    # Unexpected event type
    status 400
    return
  end

  status 200
end

from flask import Flask, request, abort
import json
app = Flask(__name__)


@app.route('/my/webhook/url', methods=['POST'])
def receive_webhook():
    payload = request.data
    event = json.loads(payload)

    if event['type'] == 'business.created':
        business = event['data']['object']
        print('Business created!')
    elif event['type'] == 'business.updated':
        business = event['data']['object']
        print('Business updated!')
    # ... handle other event types
    else:
        # Unexpected event type
        abort(400)

    return ''

const bodyParser = require('body-parser');
const express = require('express');
const app = express();

app.post('/my/webhook/url', (req, res) => {
  // If here, the signature is valid.
  var event = req.body;

  switch (event.type) {
  case 'business.created':
    business = event.data.object;
    console.log(`Business ${business.id} created!`);
    break;
  case 'business.updated':
    business = event.data.object;
    console.log(`Business ${business.id} updated!`);
    break;
  // ... handle other event types.
  default:
    res.status(400).send('Unexpected event type');
    return;
  }

  res.status(200)
});


// Start the server
const port = 3000;
app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

Event Overview

A Webhook request consists of an Event object payload that Middesk will send via an HTTP POST request to your URL endpoint.

The request contains all the relevant information about how the Business object in question was updated, including the type of event and the data associated with it.

The id of the main object will be included in the headers with the key X-Correlation-Id. This allows for easily mapping together related Webhook events when only the headers are available.

Event Object

An Event object payload contains the following fields:

{
  "object": "event",
  "id": "f215a707-655e-400f-84e6-fbb949f5612a",
  "type": "business.created",
  "data": {
    "object": {
      "id": "0f86dab5-8195-4b95-b3c0-19deaeba2a8e",
      "tin": null,
      "name": "A Company",
      "tags": [],
      "names": [],
      "domain": null,
      "object": "business",
      "review": null,
      "status": "open",
      "orders": [
        {
          "id": "d9e4076c-25d7-4a93-917b-66568459cbc4",
          "object": "order",
          "status": "pending",
          "product": "identity",
          "created_at": "2020-01-02T23:54:48.180Z",
          "updated_at": "2020-01-02T23:54:48.180Z",
          "completed_at": null
        }
      ],
      "summary": null,
      "website": null,
      "officers": [],
      "addresses": [],
      "formation": null,
      "watchlist": null,
      "created_at": "2020-01-02T23:54:48.154Z",
      "updated_at": "2020-01-02T23:54:48.154Z",
      "external_id": null,
      "phone_numbers": [],
      "registrations": []
    }
  },
  "created_at": "2020-01-02T23:54:48.239Z"
}

🚧

Note: Middesk will add new events in the future. If there are specific events that you would like prioritized, please contact [email protected].

Types of Webhook Events

Today, Webhook requests consist of the following types:

Monitor-specific Webhook Events

If you create a Monitor for a specific event_type, you will receive one or more of the following webhook requests.

Webhook Retry Policy

If a webhook fails to make the request to the client's server, it will attempt up to make the request up to 10 times until it has a successful request. The first request will be delayed 1 - 30 seconds from the initial request. If that request fails, the next request will occur 1 - 30 seconds after that. Every subsequent request will exponentially backoff. The final request will be made approximately 3 days from when the first request was made.

IP Addresses for Webhooks

Below is the list of IP addresses that Middesk sends webhook requests from:

• 35.239.59.102
• 35.192.63.74
• 104.198.38.1

Checking Webhook Signatures

Middesk can optionally sign the Webhook requests that it sends to your endpoints. We do so by including a signature in the request's X-Middesk-Signature-256 header. This header allows you to verify that the requests were sent by Middesk, not by a third party.

Security Note: Importance of Verifying HMAC Signature with Raw Request Body

In the context of securing webhooks, it is critical to verify the HMAC (Hash-based Message Authentication Code) signature using the raw request body before parsing it as JSON or any other format. This process is essential for the following reasons:

Data Integrity: The HMAC signature is calculated using the exact bytes of the request body. Parsing the body can inadvertently alter the byte sequence—for instance, by changing the encoding, reformatting JSON structures, or trimming whitespaces. Any such alteration, even if semantically equivalent in a parsed form, will lead to a different byte sequence and thus a different HMAC signature, causing signature verification to fail.

Preventing Parsing Vulnerabilities: Parsing the request body (especially when it contains JSON) can expose the application to various attacks if the input is not trusted. By verifying the HMAC signature against the raw body, you ensure that the payload has not been tampered with and is exactly what the sender intended. Only verified and thus trusted payloads should be parsed and processed further.

Before you can verify signatures, you'll need to set a secret via the Webhooks API.

Signatures are generated by Middesk using a hash-based message authentication code (HMAC) with SHA-256. To check the signature, follow these steps:

Step 1: Extract the signature from the header

Read the value from the X-Middesk-Signature-256 header.

Step 2: Prepare the expected signature

Compute an HMAC with the SHA-256 hash function. Use the provided secret as the key, and use the original request body as the message.

Step 3: Compare the Signatures

Compare the signature extracted from the header to the expected signature that you computed using a secure, constant time string comparison method.

require 'openssl'

secret = 'sec_...'

def verify(payload, signature)
  digest = OpenSSL::Digest.new('sha256')
  expected = OpenSSL::HMAC.hexdigest(digest, secret, payload)
  OpenSSL.secure_compare(expected, signature)
end

post '/my/webhook/url' do
  payload = request.body.read
  sig_header = request.env['X_MIDDESK_SIGNATURE_256']

  unless verify(payload, sig_header)
    # Invalid signature
    status 400
    return
  end
    
  event = JSON.parse(payload)
  
  case event.type
  when 'business.created'
    business = event.data.object
    puts 'Business created!'
  when 'business.updated'
    business = event.data.object
    puts 'Business updated!'
  # ... handle other event types
  else
    # Unexpected event type
    status 400
    return
  end

  status 200
end

from flask import Flask, request, abort
import hmac
import hashlib
import json
app = Flask(__name__)


secret = 'sec_...'


def verify(payload, signature):
    expected = hmac.new(secret, payload, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature)


@app.route('/my/webhook/url', methods=['POST'])
def receive_webhook():
    payload = request.data
    sig_header = request.headers['X-Middesk-Signature-256']

    if not verify(payload, sig_header):
        # Invalid signature
        abort(400)

    event = json.loads(payload)

    if event['type'] == 'business.created':
        business = event['data']['object']
        print('Business created!')
    elif event['type'] == 'business.updated':
        business = event['data']['object']
        print('Business updated!')
    # ... handle other event types
    else:
        # Unexpected event type
        abort(400)

    return ''

const bodyParser = require('body-parser');
const crypto = require('crypto');
const express = require('express');
const app = express();


const MIDDLEWARE_WEBHOOK_SECRET = process.env.MIDDESK_WEBHOOK_SECRET;

// Route-specific middleware for raw body parsing with signature verification.
const verifyMiddleware = bodyParser.json({
  verify: function(req, res, buf, encoding) {
    const signatureHeader = req.get('X-Middesk-Signature-256');

    const signature = crypto
      .createHmac('sha256', MIDDLEWARE_WEBHOOK_SECRET)
      .update(buf)
      .digest('hex');

    if (signature !== signatureHeader) {
      res.status(401).send('Invalid signature.');
      throw new Error(`Invalid signature. Got ${signature}. Expected ${signatureHeader}`);
    }
  }
});

// Apply the verification middleware only to this specific route.
app.post('/my/webhook/url', verifyMiddleware, (req, res) => {
  // If here, the signature is valid.
  var event = req.body;

  switch (event.type) {
  case 'business.created':
    business = event.data.object;
    console.log(`Business ${business.id} created!`);
    break;
  case 'business.updated':
    business = event.data.object;
    console.log(`Business ${business.id} updated!`);
    break;
  // ... handle other event types.
  default:
    res.status(400).send('Unexpected event type');
    return;
  }

  res.status(200)
});


// Start the server
const port = 3000;
app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

Authenticating Webhooks using Mutual TLS

Another way to authenticate Middesk webhook requests is to verify a secure connection with the client using Mutual Transport Layer Security (Mutual TLS).

In a typical TLS connection, only the server's identity is authenticated by presenting a certificate to the client. The client verifies the server's certificate using a trusted 3rd party certificate authority. With Mutual TLS, the client will also present a certificate of its own to the server. This requires you to configure your server to request and verify a client certificate during the TLS connection handshake.

Middesk's webhook client certificate is issued by the DigiCert Assured ID Root G2 Certificate Authority which is a well-known certificate authority which is already configured on most operating systems by default but can be downloaded directly here or found among the list of DigiCert root certificates.

Our chosen certificate authority issues many certificates so it is crucial to verify that the client is indeed Middesk! Your server should verify that the client certificate has the following Subject Distinguished Name (Subject DN) fields before handling a webhook request:

O=Middesk\, Inc.

and

CN=webhooks.middesk.com

The first ensures that the client certificate was issued to Middesk, Inc., as verified by the certificate authority. The second ensures that the client certificate is being used as intended.

It is quite common for client TLS connection termination and HTTP request routing to be performed by a reverse proxy server. In this case, you must configure this proxy server to ask for and verify a client certificate. Then, before routing the webhook request to your backend server application, verify that the client certificate subject matches the one above.

Authenticating Webhooks using OAuth Access Tokens Issued by an OpenID Connect Identity Provider (Advanced)

Application Security is a paramount concern for enterprises that demand high-security standards and have complex internal infrastructures. The motivation for using OAuth access tokens issued by an OpenID Connect Identity Provider for webhook authentication, especially for large enterprises, can be understood in the context of the limitations presented by other authentication methods:

• HMAC with Shared Secret: While HMAC signatures provide a level of security, they rely on a shared secret. This approach does not align with the security protocols of some customers who require a more secure, non-shared method due to the inherent risks associated with distributing and managing shared secrets.
• Mutual TLS: Mutual TLS elevates security by using a client certificate, avoiding shared secrets. However, this method can be challenging for organizations with intricate network architectures, such as those that utilize edge proxies. These proxies may not support client certificate authentication or make it overly complex to verify and relay the client TLS certificate information to the backend application.

In contrast, utilizing OAuth Access Tokens via OpenID Connect is tailored for customers who encounter the limitations mentioned above. The key motivations for adopting this method are:

• Enhanced Security: Leveraging internal OpenID Connect Identity Providers allows for the issuance of trusted tokens without sharing secrets externally.
• Separation of Concerns: Security responsibilities are divided between teams. The Identity Provider management is handled by a dedicated security team, while application teams focus solely on verifying access tokens intended for their applications.
• Scalability and Trust: Large enterprises often operate with an ecosystem of internal applications. Utilizing an internal IdP to issue tokens ensures that the tokens are trusted across all applications within the enterprise.
• Specificity and Precision: Tokens can be scoped precisely to the intended application by specifying the resource URI, thereby enhancing control over access and reducing the attack surface.
• Compliance with Internal Policies: For organizations with stringent internal security policies and compliance requirements, using an internal IdP for token issuance aligns with their governance models.

By adopting OAuth access tokens for webhook authentication, enterprises can thus ensure a secure, scalable, and compliant integration environment that aligns with their complex and high-security infrastructure needs.

Authentication Workflow with OAuth Access Tokens

The authentication process using OAuth Access Tokens operates in three main stages:

1. Token Request: Our Webhook Worker initiates the process by requesting an access token from your Identity Provider. This is done by sending a secure request to the Identity Provider's token endpoint, which includes the Webhook Worker's credentials and the specified scope for the access token (a URI for your webhook endpoint).
2. Token Issuance: Upon validating the Webhook Worker's credentials and token request, your Identity Provider responds by issuing an access token. This token is digitally signed and contains the necessary claims that identify the permissions granted to our Webhook Worker. The token often often has a short time period in which it may be reused, in which case the Webhook Worker can be configured to cache the token for a short period of time to use with near-future webhook events, saving time and resources by skipping steps 1 and 2 if possible.
3. Webhook Invocation: With the access token obtained, our Webhook Worker specifies it in the authorization header and sends the request to your API endpoint which is configured to handle incoming webhook requests. The header is of the form: Authorization: Bearer {ACCESS_TOKEN}. Your API endpoint will then validate the access token. Though the precise method by which it is validated is not specified in the OpenID Connect specification, this is typically done by checking the token's signature against the known public keys used by the Identity Provider when issuing tokens. Please refer to your enterprise's internal documentation on how to validate these access tokens.

This workflow ensures that each webhook request is authenticated in a secure and standardized manner, leveraging the robustness of OAuth 2.0 and OpenID Connect protocols.

Requirements for OAuth Access Token Authentication Setup

To implement webhook authentication using OAuth Access Tokens issued by an OpenID Connect Identity Provider, the following setup is required:

1. OpenID Connect Provider Client Registration: Register an application to represent Middesk's Webhook Worker with your enterprise's internal OpenID Connect Provider. This registration process should result in obtaining a Client ID that uniquely identifies Middesk's Webhook Worker within your enterprise's ecosystem.
   1. JWK Set URL: As part of the client registration setup, you will need to provide a JSON Web Key Set (JWKS) URL. This URL will be utilized by your enterprise's Identity Provider to validate the signature of the the Middesk Webhook Worker's authentication credentials when it requests an access token for your API endpoint. Middesk's JWK Set URL is: https://api.middesk.com/v1/webhooks/oidc_keys
   2. Client ID Acquisition: Obtain the Client ID that was generated during the client registration process. This identifier is essential for the OAuth token request and will be used to authenticate the Middesk Webhook Worker to your Identity Provider.
2. IdP Token Endpoint: Locate the Token Endpoint URL of your enterprise's Identity Provider. This endpoint is where Middesk's Webhook Worker will send a request to obtain an OAuth access token.
3. Your API Resource URI: Specify the exact Resource URI of your API endpoint or internal application which receives and processes webhook events. The Middesk Webhook Worker uses this URI to indicate to your enterprise's Identity Provider that the access token being requested is intended for your specific API endpoint.
4. Access Token Time-to-Live (TTL): Define the TTL for the access tokens in seconds. This duration determines how long an access token is valid before it expires and a new one must be requested. A suitable TTL is crucial for balancing security with practicality—short enough to mitigate risk, but long enough to ensure uninterrupted service. The Middesk Webhook Worker will use this value to determine how long to store cached access tokens for reuse.
5. IdP Trust Configuration: Ensure that your API is configured to trust your enterprise's internal OpenID Connect Provider. Neither OAuth 2.0 nor OpenID Connect specify the format of access tokens or how to validate them, but this typically involves setting up the service to validate the issuer of the access token and to trust access tokens signed with keys provided by the JWKS URL. Please refer to your enterprise's internal documentation on how to validate these access tokens.

By meeting these requirements, an enterprise can establish a secure and efficient authentication process for webhooks that aligns with their internal security protocols and simplifies the responsibilities of their application development teams.

Configuring a Webhook to use OAuth Access Tokens

Once you have met all of the requirements above, you are ready to configure a Webhook to use access tokens. For this, there are 4 additional fields which must be specified in your API request to create or update a Webhook (these may not be specified via the Webhooks UI for now). All 4 fields must be specified together.