Skip to main content

Integrating External APIs

Learn how to integrate Stack9 with external services and APIs. This guide covers REST API connectors, authentication methods, error handling, and real-world integration patterns for common third-party services.

What You'll Build

In this guide, you'll integrate with multiple external services:

  • Payment Gateway (Stripe-like API) with Bearer token auth
  • Email Service (SendGrid-like) with API key auth
  • Address Lookup (Google Maps-like) with query parameter auth
  • CRM System (Salesforce-like) with OAuth2
  • SMS Service (Twilio-like) with Basic auth

Time to complete: 60-75 minutes

Prerequisites

  • Completed Building Workflows guide
  • Understanding of REST APIs
  • API credentials for services you want to integrate (or use test/sandbox APIs)

Understanding Stack9 Connectors

Connectors in Stack9 provide a standardized way to integrate with external services:

Connector Types

TypeUse CaseExample Services
REST_APIREST/HTTP APIsMost modern APIs
AWS_OPENSEARCHAWS OpenSearchElasticsearch alternative
AWS_S3File storageAmazon S3, MinIO
AZURE_BLOBFile storageAzure Blob Storage
POSTGRESQLExternal databasesPostgreSQL databases
MONGODBNoSQL databasesMongoDB Atlas
REDISCachingRedis Cloud
GRAPHQLGraphQL APIsGitHub, Shopify
SENDGRIDEmail serviceSendGrid
OPEN_APIOpenAPI/SwaggerAny OpenAPI spec
AWS_DYNAMODBNoSQL tablesDynamoDB
ELASTICSEARCHSearch engineElasticsearch

Authentication Methods

MethodHeader FormatUse Case
BEARER_TOKENAuthorization: Bearer TOKENModern APIs (OAuth2)
API_KEYX-API-Key: KEY or custom headerSimple auth
BASIC_AUTHAuthorization: Basic BASE64Legacy systems
OAUTH2OAuth2 flowGoogle, Facebook, etc.
CUSTOMCustom headersSpecial cases

Step 1: Create Payment Gateway Connector

Create src/connectors/payment_gateway.json:

{
"name": "payment_gateway",
"label": "Payment Gateway",
"description": "Stripe-compatible payment processing",
"type": "REST_API",
"config": {
"baseUrl": "https://api.payment-gateway.com/v1",
"auth": {
"type": "BEARER_TOKEN",
"token": "%%PAYMENT_GATEWAY_API_KEY%%"
},
"headers": {
"Content-Type": "application/json",
"Accept": "application/json"
},
"timeout": 30000
}
}

Environment Variable:

Add to .env:

PAYMENT_GATEWAY_API_KEY=sk_test_your_secret_key_here

Using the Connector in Action Type

Create src/action-types/processPayment.ts:

import { Record, Number, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';

const ProcessPaymentParams = Record({
amount: Number,
currency: String,
customer_email: String,
order_id: Number,
});

export class ProcessPayment extends S9AutomationActionType {
key = 'process_payment';
name = 'Process Payment';

async exec(params: any) {
const validated = ProcessPaymentParams.check(params);
const { connectors, logger } = this.context;

try {
const paymentGateway = connectors['payment_gateway'];

// Create payment intent
const paymentIntent = await paymentGateway.call({
method: 'POST',
path: '/payment_intents',
body: {
amount: Math.round(validated.amount * 100), // Convert to cents
currency: validated.currency,
customer_email: validated.customer_email,
metadata: {
order_id: validated.order_id,
},
},
});

logger.info(
`Payment intent created: ${paymentIntent.id} for order ${validated.order_id}`
);

// Confirm payment
const confirmedPayment = await paymentGateway.call({
method: 'POST',
path: `/payment_intents/${paymentIntent.id}/confirm`,
body: {
payment_method: 'pm_card_visa', // In real app, from customer
},
});

if (confirmedPayment.status === 'succeeded') {
return {
success: true,
transaction_id: confirmedPayment.id,
amount: validated.amount,
message: 'Payment processed successfully',
};
} else {
return {
success: false,
error: `Payment status: ${confirmedPayment.status}`,
};
}
} catch (error) {
logger.error(`Payment processing failed: ${error.message}`);

// Parse error response
const errorMessage =
error.response?.data?.error?.message || error.message;

return {
success: false,
error: errorMessage,
message: 'Payment failed',
};
}
}
}

Step 2: Create Email Service Connector

Create src/connectors/email_service.json:

{
"name": "email_service",
"label": "Email Service",
"description": "SendGrid-compatible email delivery",
"type": "REST_API",
"config": {
"baseUrl": "https://api.sendgrid.com/v3",
"auth": {
"type": "API_KEY",
"header": "Authorization",
"prefix": "Bearer",
"value": "%%SENDGRID_API_KEY%%"
},
"headers": {
"Content-Type": "application/json"
},
"timeout": 10000
}
}

Environment Variable:

SENDGRID_API_KEY=SG.your_sendgrid_api_key_here

Using Email Connector

Create src/action-types/sendEmail.ts:

import { Record, String, Optional } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';

const SendEmailParams = Record({
to: String,
subject: String,
template: String,
data: Optional(Record({})),
});

export class SendEmail extends S9AutomationActionType {
key = 'send_email';
name = 'Send Email';

async exec(params: any) {
const validated = SendEmailParams.check(params);
const { connectors, logger } = this.context;

try {
const emailService = connectors['email_service'];

// Fetch template from database
const template = await this.context.db('email_template')
.where({ key: validated.template })
.first();

if (!template) {
throw new Error(`Email template '${validated.template}' not found`);
}

// Replace template variables
let htmlContent = template.html_content;
let textContent = template.text_content;

if (validated.data) {
for (const [key, value] of Object.entries(validated.data)) {
const regex = new RegExp(`{{${key}}}`, 'g');
htmlContent = htmlContent.replace(regex, String(value));
textContent = textContent.replace(regex, String(value));
}
}

// Send email
const response = await emailService.call({
method: 'POST',
path: '/mail/send',
body: {
personalizations: [
{
to: [{ email: validated.to }],
},
],
from: {
email: 'noreply@yourapp.com',
name: 'Your App Name',
},
subject: validated.subject,
content: [
{
type: 'text/plain',
value: textContent,
},
{
type: 'text/html',
value: htmlContent,
},
],
},
});

logger.info(`Email sent to ${validated.to}: ${validated.subject}`);

return {
success: true,
message_id: response.headers['x-message-id'],
message: 'Email sent successfully',
};
} catch (error) {
logger.error(`Email sending failed: ${error.message}`);
throw error;
}
}
}

Step 3: Create Address Lookup Connector

Create src/connectors/address_lookup.json:

{
"name": "address_lookup",
"label": "Address Lookup Service",
"description": "Google Maps-compatible address geocoding",
"type": "REST_API",
"config": {
"baseUrl": "https://maps.googleapis.com/maps/api",
"auth": {
"type": "QUERY_PARAM",
"param": "key",
"value": "%%GOOGLE_MAPS_API_KEY%%"
},
"headers": {
"Accept": "application/json"
},
"timeout": 5000
}
}

Environment Variable:

GOOGLE_MAPS_API_KEY=your_google_maps_api_key

Using Address Lookup

Create src/action-types/validateAddress.ts:

import { Record, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';

export class ValidateAddress extends S9AutomationActionType {
key = 'validate_address';
name = 'Validate Address';

async exec(params: any) {
const { address } = params;
const { connectors, logger } = this.context;

try {
const addressLookup = connectors['address_lookup'];

// Geocode address
const response = await addressLookup.call({
method: 'GET',
path: '/geocode/json',
params: {
address: address,
},
});

if (response.status === 'OK' && response.results.length > 0) {
const result = response.results[0];

// Extract address components
const components = result.address_components;
const parsed = {
formatted_address: result.formatted_address,
street_number: this.getComponent(components, 'street_number'),
route: this.getComponent(components, 'route'),
city: this.getComponent(components, 'locality'),
state: this.getComponent(components, 'administrative_area_level_1'),
postal_code: this.getComponent(components, 'postal_code'),
country: this.getComponent(components, 'country'),
latitude: result.geometry.location.lat,
longitude: result.geometry.location.lng,
};

logger.info(`Address validated: ${parsed.formatted_address}`);

return {
success: true,
valid: true,
parsed_address: parsed,
};
} else {
return {
success: true,
valid: false,
error: 'Address not found',
};
}
} catch (error) {
logger.error(`Address validation failed: ${error.message}`);
throw error;
}
}

private getComponent(components: any[], type: string): string {
const component = components.find((c) => c.types.includes(type));
return component?.long_name || '';
}
}

Step 4: Create SMS Service Connector (Basic Auth)

Create src/connectors/sms_service.json:

{
"name": "sms_service",
"label": "SMS Service",
"description": "Twilio-compatible SMS delivery",
"type": "REST_API",
"config": {
"baseUrl": "https://api.twilio.com/2010-04-01",
"auth": {
"type": "BASIC_AUTH",
"username": "%%TWILIO_ACCOUNT_SID%%",
"password": "%%TWILIO_AUTH_TOKEN%%"
},
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"timeout": 15000
}
}

Environment Variables:

TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=your_auth_token_here

Using SMS Connector

Create src/action-types/sendSMS.ts:

import { Record, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
import querystring from 'querystring';

const SendSMSParams = Record({
to: String,
message: String,
});

export class SendSMS extends S9AutomationActionType {
key = 'send_sms';
name = 'Send SMS';

async exec(params: any) {
const validated = SendSMSParams.check(params);
const { connectors, logger } = this.context;

try {
const smsService = connectors['sms_service'];
const accountSid = process.env.TWILIO_ACCOUNT_SID;

// Twilio uses form-encoded data
const body = querystring.stringify({
To: validated.to,
From: process.env.TWILIO_PHONE_NUMBER,
Body: validated.message,
});

const response = await smsService.call({
method: 'POST',
path: `/Accounts/${accountSid}/Messages.json`,
body: body,
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
});

logger.info(`SMS sent to ${validated.to}`);

return {
success: true,
message_sid: response.sid,
message: 'SMS sent successfully',
};
} catch (error) {
logger.error(`SMS sending failed: ${error.message}`);
throw error;
}
}
}

Step 5: Create CRM Connector (OAuth2)

Create src/connectors/crm_api.json:

{
"name": "crm_api",
"label": "CRM API",
"description": "Salesforce-compatible CRM integration",
"type": "REST_API",
"config": {
"baseUrl": "https://api.crm-system.com/v2",
"auth": {
"type": "OAUTH2",
"tokenUrl": "https://api.crm-system.com/oauth/token",
"clientId": "%%CRM_CLIENT_ID%%",
"clientSecret": "%%CRM_CLIENT_SECRET%%",
"scope": "contacts:write contacts:read"
},
"headers": {
"Content-Type": "application/json",
"Accept": "application/json"
},
"timeout": 30000
}
}

Environment Variables:

CRM_CLIENT_ID=your_client_id
CRM_CLIENT_SECRET=your_client_secret

Using CRM Connector

Create src/action-types/syncContactToCRM.ts:

import { Record, Number } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';

export class SyncContactToCRM extends S9AutomationActionType {
key = 'sync_contact_to_crm';
name = 'Sync Contact to CRM';

async exec(params: any) {
const { customer_id } = params;
const { db, connectors, logger } = this.context;

try {
// Fetch customer
const customer = await db('customer')
.where({ id: customer_id })
.first();

if (!customer) {
throw new Error(`Customer ${customer_id} not found`);
}

const crmApi = connectors['crm_api'];

// Check if contact already exists in CRM
const existingContacts = await crmApi.call({
method: 'GET',
path: '/contacts',
params: {
email: customer.email,
},
});

if (existingContacts.data && existingContacts.data.length > 0) {
// Update existing contact
const contactId = existingContacts.data[0].id;

await crmApi.call({
method: 'PUT',
path: `/contacts/${contactId}`,
body: {
first_name: customer.first_name,
last_name: customer.last_name,
email: customer.email,
phone: customer.phone,
company: customer.company,
status: customer.status,
custom_fields: {
stack9_customer_id: customer.id,
last_synced: new Date().toISOString(),
},
},
});

logger.info(`Updated CRM contact ${contactId} for customer ${customer.email}`);

return {
success: true,
crm_contact_id: contactId,
action: 'updated',
};
} else {
// Create new contact
const newContact = await crmApi.call({
method: 'POST',
path: '/contacts',
body: {
first_name: customer.first_name,
last_name: customer.last_name,
email: customer.email,
phone: customer.phone,
company: customer.company,
status: customer.status,
custom_fields: {
stack9_customer_id: customer.id,
created_in_stack9: new Date().toISOString(),
},
},
});

logger.info(`Created CRM contact ${newContact.id} for customer ${customer.email}`);

return {
success: true,
crm_contact_id: newContact.id,
action: 'created',
};
}
} catch (error) {
logger.error(`CRM sync failed: ${error.message}`);
throw error;
}
}
}

Step 6: Error Handling Patterns

Pattern 1: Retry with Exponential Backoff

export class RetryableAction extends S9AutomationActionType {
key = 'retryable_action';
name = 'Retryable Action';

async exec(params: any) {
const maxRetries = 3;
let lastError: Error;

for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
const result = await this.makeApiCall(params);
return result;
} catch (error) {
lastError = error;

if (attempt < maxRetries) {
// Exponential backoff: 1s, 2s, 4s
const delay = Math.pow(2, attempt - 1) * 1000;
this.context.logger.warn(
`Attempt ${attempt} failed, retrying in ${delay}ms`
);
await this.sleep(delay);
}
}
}

throw new Error(`Failed after ${maxRetries} attempts: ${lastError.message}`);
}

private async makeApiCall(params: any) {
// Your API call here
}

private sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
}

Pattern 2: Circuit Breaker

class CircuitBreaker {
private failures = 0;
private lastFailureTime = 0;
private readonly threshold = 5;
private readonly timeout = 60000; // 1 minute

async call<T>(fn: () => Promise<T>): Promise<T> {
// Check if circuit is open
if (this.failures >= this.threshold) {
const timeSinceLastFailure = Date.now() - this.lastFailureTime;
if (timeSinceLastFailure < this.timeout) {
throw new Error('Circuit breaker is open - too many failures');
}
// Reset after timeout
this.failures = 0;
}

try {
const result = await fn();
this.failures = 0; // Reset on success
return result;
} catch (error) {
this.failures++;
this.lastFailureTime = Date.now();
throw error;
}
}
}

export class ResilientAction extends S9AutomationActionType {
private circuitBreaker = new CircuitBreaker();

async exec(params: any) {
return this.circuitBreaker.call(async () => {
const { connectors } = this.context;
return await connectors['external_api'].call({
method: 'GET',
path: '/data',
});
});
}
}

Pattern 3: Timeout Handling

async function withTimeout<T>(
promise: Promise<T>,
timeoutMs: number,
errorMessage = 'Operation timed out'
): Promise<T> {
let timeoutHandle: NodeJS.Timeout;

const timeoutPromise = new Promise<T>((_, reject) => {
timeoutHandle = setTimeout(() => {
reject(new Error(errorMessage));
}, timeoutMs);
});

try {
return await Promise.race([promise, timeoutPromise]);
} finally {
clearTimeout(timeoutHandle!);
}
}

export class TimeoutAction extends S9AutomationActionType {
async exec(params: any) {
const { connectors } = this.context;

try {
// 10 second timeout
const result = await withTimeout(
connectors['slow_api'].call({ method: 'GET', path: '/data' }),
10000,
'API call timed out after 10 seconds'
);

return { success: true, data: result };
} catch (error) {
if (error.message.includes('timed out')) {
this.context.logger.error('API timeout - using cached data');
// Fallback to cached data
return { success: true, cached: true, data: await this.getCachedData() };
}
throw error;
}
}
}

Pattern 4: Rate Limiting

class RateLimiter {
private requests: number[] = [];
private readonly limit: number;
private readonly windowMs: number;

constructor(limit: number, windowMs: number) {
this.limit = limit;
this.windowMs = windowMs;
}

async throttle(): Promise<void> {
const now = Date.now();

// Remove old requests outside the window
this.requests = this.requests.filter((time) => now - time < this.windowMs);

if (this.requests.length >= this.limit) {
const oldestRequest = this.requests[0];
const waitTime = this.windowMs - (now - oldestRequest);
await new Promise((resolve) => setTimeout(resolve, waitTime));
return this.throttle(); // Recursive retry
}

this.requests.push(now);
}
}

export class RateLimitedAction extends S9AutomationActionType {
// 100 requests per minute
private rateLimiter = new RateLimiter(100, 60000);

async exec(params: any) {
await this.rateLimiter.throttle();

const { connectors } = this.context;
return await connectors['api'].call({
method: 'GET',
path: '/resource',
});
}
}

Step 7: Testing External Integrations

Test Payment Processing

# Test payment action
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "process_payment",
"params": {
"amount": 99.99,
"currency": "USD",
"customer_email": "test@example.com",
"order_id": 1
}
}'

Test Email Sending

# Test email action
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "send_email",
"params": {
"to": "recipient@example.com",
"subject": "Test Email",
"template": "welcome",
"data": {
"first_name": "John"
}
}
}'

Test Address Validation

# Test address lookup
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "validate_address",
"params": {
"address": "1600 Amphitheatre Parkway, Mountain View, CA"
}
}'

Step 8: Advanced Patterns

Pattern 1: Webhook Signature Verification

import crypto from 'crypto';

export class VerifyWebhook extends S9AutomationActionType {
key = 'verify_webhook';
name = 'Verify Webhook Signature';

async exec(params: any) {
const { payload, signature, secret } = params;

// Calculate expected signature
const expectedSignature = crypto
.createHmac('sha256', secret)
.update(JSON.stringify(payload))
.digest('hex');

const isValid = crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(expectedSignature)
);

if (!isValid) {
throw new Error('Invalid webhook signature');
}

return {
success: true,
valid: true,
message: 'Webhook signature verified',
};
}
}

Pattern 2: Pagination Handling

export class FetchAllRecords extends S9AutomationActionType {
async exec(params: any) {
const { connectors, logger } = this.context;
const api = connectors['external_api'];

let allRecords = [];
let page = 1;
let hasMore = true;

while (hasMore) {
logger.info(`Fetching page ${page}`);

const response = await api.call({
method: 'GET',
path: '/records',
params: {
page: page,
per_page: 100,
},
});

allRecords = allRecords.concat(response.data);
hasMore = response.has_more;
page++;

// Safety limit
if (page > 100) {
logger.warn('Reached maximum page limit (100)');
break;
}
}

logger.info(`Fetched total of ${allRecords.length} records`);

return {
success: true,
total_records: allRecords.length,
records: allRecords,
};
}
}

Pattern 3: Bulk API Operations

export class BulkSync extends S9AutomationActionType {
async exec(params: any) {
const { db, connectors, logger } = this.context;

// Fetch records that need syncing
const records = await db('customer')
.where('needs_sync', true)
.limit(1000);

logger.info(`Syncing ${records.length} customers`);

// Batch records (API allows max 100 per request)
const batchSize = 100;
const batches = [];

for (let i = 0; i < records.length; i += batchSize) {
batches.push(records.slice(i, i + batchSize));
}

const api = connectors['external_api'];
let successCount = 0;
let errorCount = 0;

for (const [index, batch] of batches.entries()) {
try {
logger.info(`Processing batch ${index + 1}/${batches.length}`);

await api.call({
method: 'POST',
path: '/bulk/customers',
body: {
customers: batch.map((c) => ({
email: c.email,
name: `${c.first_name} ${c.last_name}`,
status: c.status,
})),
},
});

// Mark as synced
await db('customer')
.whereIn('id', batch.map((c) => c.id))
.update({ needs_sync: false, last_synced: new Date() });

successCount += batch.length;
} catch (error) {
logger.error(`Batch ${index + 1} failed: ${error.message}`);
errorCount += batch.length;
}
}

return {
success: true,
synced: successCount,
failed: errorCount,
total: records.length,
};
}
}

Best Practices

Security

  1. Never hardcode credentials - Always use environment variables
  2. Validate webhook signatures - Verify requests from external services
  3. Use HTTPS only - Never send credentials over HTTP
  4. Rotate API keys regularly - Implement key rotation policies
  5. Limit API key permissions - Use least privilege principle

Error Handling

  1. Implement retries - Transient failures are common with APIs
  2. Use circuit breakers - Prevent cascading failures
  3. Set appropriate timeouts - Don't let operations hang forever
  4. Log all errors - Include request/response details
  5. Provide fallbacks - Gracefully degrade when APIs are down

Performance

  1. Cache responses - Cache frequently accessed data
  2. Use batch operations - Reduce number of API calls
  3. Implement rate limiting - Respect API rate limits
  4. Paginate large datasets - Don't fetch everything at once
  5. Use async operations - Don't block on long-running calls

Monitoring

  1. Track API usage - Monitor request counts and quotas
  2. Log response times - Identify slow APIs
  3. Alert on failures - Set up alerts for critical integrations
  4. Monitor rate limits - Don't hit rate limit ceilings
  5. Track costs - Many APIs charge per request

Troubleshooting

Authentication Failures

Problem: 401 Unauthorized responses

Solutions:

  1. Verify API key/token is correct
  2. Check environment variable is set
  3. Ensure auth header format matches API requirements
  4. Check if token has expired (OAuth2)
  5. Verify API key has required permissions

Timeout Errors

Problem: Requests timing out

Solutions:

  1. Increase timeout in connector config
  2. Check if external API is down (status page)
  3. Implement retry logic with exponential backoff
  4. Use async operations for long requests
  5. Contact API support if persistent

Rate Limit Exceeded

Problem: 429 Too Many Requests

Solutions:

  1. Implement rate limiting in your code
  2. Add delays between requests
  3. Use batch endpoints when available
  4. Cache responses to reduce calls
  5. Upgrade API plan if needed

Invalid Response Format

Problem: Unexpected API response structure

Solutions:

  1. Check API documentation for response format
  2. Log full response for debugging
  3. Handle different response formats
  4. Check API version hasn't changed
  5. Validate response schema

SSL/TLS Errors

Problem: Certificate verification failures

Solutions:

  1. Ensure correct base URL (https://)
  2. Check server certificate is valid
  3. Update Node.js version if outdated
  4. For testing only: disable cert verification (not production!)
  5. Contact API provider about certificate issues

Next Steps

You've mastered external API integration! Continue learning: