Integrating External APIs
Learn how to integrate Stack9 with external services and APIs. This guide covers REST API connectors, authentication methods, error handling, and real-world integration patterns for common third-party services.
What You'll Build
In this guide, you'll integrate with multiple external services:
- Payment Gateway (Stripe-like API) with Bearer token auth
- Email Service (SendGrid-like) with API key auth
- Address Lookup (Google Maps-like) with query parameter auth
- CRM System (Salesforce-like) with OAuth2
- SMS Service (Twilio-like) with Basic auth
Time to complete: 60-75 minutes
Prerequisites
- Completed Building Workflows guide
- Understanding of REST APIs
- API credentials for services you want to integrate (or use test/sandbox APIs)
Understanding Stack9 Connectors
Connectors in Stack9 provide a standardized way to integrate with external services:
Connector Types
| Type | Use Case | Example Services |
|---|---|---|
REST_API | REST/HTTP APIs | Most modern APIs |
AWS_OPENSEARCH | AWS OpenSearch | Elasticsearch alternative |
AWS_S3 | File storage | Amazon S3, MinIO |
AZURE_BLOB | File storage | Azure Blob Storage |
POSTGRESQL | External databases | PostgreSQL databases |
MONGODB | NoSQL databases | MongoDB Atlas |
REDIS | Caching | Redis Cloud |
GRAPHQL | GraphQL APIs | GitHub, Shopify |
SENDGRID | Email service | SendGrid |
OPEN_API | OpenAPI/Swagger | Any OpenAPI spec |
AWS_DYNAMODB | NoSQL tables | DynamoDB |
ELASTICSEARCH | Search engine | Elasticsearch |
Authentication Methods
| Method | Header Format | Use Case |
|---|---|---|
BEARER_TOKEN | Authorization: Bearer TOKEN | Modern APIs (OAuth2) |
API_KEY | X-API-Key: KEY or custom header | Simple auth |
BASIC_AUTH | Authorization: Basic BASE64 | Legacy systems |
OAUTH2 | OAuth2 flow | Google, Facebook, etc. |
CUSTOM | Custom headers | Special cases |
Step 1: Create Payment Gateway Connector
Create src/connectors/payment_gateway.json:
{
"name": "payment_gateway",
"label": "Payment Gateway",
"description": "Stripe-compatible payment processing",
"type": "REST_API",
"config": {
"baseUrl": "https://api.payment-gateway.com/v1",
"auth": {
"type": "BEARER_TOKEN",
"token": "%%PAYMENT_GATEWAY_API_KEY%%"
},
"headers": {
"Content-Type": "application/json",
"Accept": "application/json"
},
"timeout": 30000
}
}
Environment Variable:
Add to .env:
PAYMENT_GATEWAY_API_KEY=sk_test_your_secret_key_here
Using the Connector in Action Type
Create src/action-types/processPayment.ts:
import { Record, Number, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
const ProcessPaymentParams = Record({
amount: Number,
currency: String,
customer_email: String,
order_id: Number,
});
export class ProcessPayment extends S9AutomationActionType {
key = 'process_payment';
name = 'Process Payment';
async exec(params: any) {
const validated = ProcessPaymentParams.check(params);
const { connectors, logger } = this.context;
try {
const paymentGateway = connectors['payment_gateway'];
// Create payment intent
const paymentIntent = await paymentGateway.call({
method: 'POST',
path: '/payment_intents',
body: {
amount: Math.round(validated.amount * 100), // Convert to cents
currency: validated.currency,
customer_email: validated.customer_email,
metadata: {
order_id: validated.order_id,
},
},
});
logger.info(
`Payment intent created: ${paymentIntent.id} for order ${validated.order_id}`
);
// Confirm payment
const confirmedPayment = await paymentGateway.call({
method: 'POST',
path: `/payment_intents/${paymentIntent.id}/confirm`,
body: {
payment_method: 'pm_card_visa', // In real app, from customer
},
});
if (confirmedPayment.status === 'succeeded') {
return {
success: true,
transaction_id: confirmedPayment.id,
amount: validated.amount,
message: 'Payment processed successfully',
};
} else {
return {
success: false,
error: `Payment status: ${confirmedPayment.status}`,
};
}
} catch (error) {
logger.error(`Payment processing failed: ${error.message}`);
// Parse error response
const errorMessage =
error.response?.data?.error?.message || error.message;
return {
success: false,
error: errorMessage,
message: 'Payment failed',
};
}
}
}
Step 2: Create Email Service Connector
Create src/connectors/email_service.json:
{
"name": "email_service",
"label": "Email Service",
"description": "SendGrid-compatible email delivery",
"type": "REST_API",
"config": {
"baseUrl": "https://api.sendgrid.com/v3",
"auth": {
"type": "API_KEY",
"header": "Authorization",
"prefix": "Bearer",
"value": "%%SENDGRID_API_KEY%%"
},
"headers": {
"Content-Type": "application/json"
},
"timeout": 10000
}
}
Environment Variable:
SENDGRID_API_KEY=SG.your_sendgrid_api_key_here
Using Email Connector
Create src/action-types/sendEmail.ts:
import { Record, String, Optional } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
const SendEmailParams = Record({
to: String,
subject: String,
template: String,
data: Optional(Record({})),
});
export class SendEmail extends S9AutomationActionType {
key = 'send_email';
name = 'Send Email';
async exec(params: any) {
const validated = SendEmailParams.check(params);
const { connectors, logger } = this.context;
try {
const emailService = connectors['email_service'];
// Fetch template from database
const template = await this.context.db('email_template')
.where({ key: validated.template })
.first();
if (!template) {
throw new Error(`Email template '${validated.template}' not found`);
}
// Replace template variables
let htmlContent = template.html_content;
let textContent = template.text_content;
if (validated.data) {
for (const [key, value] of Object.entries(validated.data)) {
const regex = new RegExp(`{{${key}}}`, 'g');
htmlContent = htmlContent.replace(regex, String(value));
textContent = textContent.replace(regex, String(value));
}
}
// Send email
const response = await emailService.call({
method: 'POST',
path: '/mail/send',
body: {
personalizations: [
{
to: [{ email: validated.to }],
},
],
from: {
email: 'noreply@yourapp.com',
name: 'Your App Name',
},
subject: validated.subject,
content: [
{
type: 'text/plain',
value: textContent,
},
{
type: 'text/html',
value: htmlContent,
},
],
},
});
logger.info(`Email sent to ${validated.to}: ${validated.subject}`);
return {
success: true,
message_id: response.headers['x-message-id'],
message: 'Email sent successfully',
};
} catch (error) {
logger.error(`Email sending failed: ${error.message}`);
throw error;
}
}
}
Step 3: Create Address Lookup Connector
Create src/connectors/address_lookup.json:
{
"name": "address_lookup",
"label": "Address Lookup Service",
"description": "Google Maps-compatible address geocoding",
"type": "REST_API",
"config": {
"baseUrl": "https://maps.googleapis.com/maps/api",
"auth": {
"type": "QUERY_PARAM",
"param": "key",
"value": "%%GOOGLE_MAPS_API_KEY%%"
},
"headers": {
"Accept": "application/json"
},
"timeout": 5000
}
}
Environment Variable:
GOOGLE_MAPS_API_KEY=your_google_maps_api_key
Using Address Lookup
Create src/action-types/validateAddress.ts:
import { Record, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
export class ValidateAddress extends S9AutomationActionType {
key = 'validate_address';
name = 'Validate Address';
async exec(params: any) {
const { address } = params;
const { connectors, logger } = this.context;
try {
const addressLookup = connectors['address_lookup'];
// Geocode address
const response = await addressLookup.call({
method: 'GET',
path: '/geocode/json',
params: {
address: address,
},
});
if (response.status === 'OK' && response.results.length > 0) {
const result = response.results[0];
// Extract address components
const components = result.address_components;
const parsed = {
formatted_address: result.formatted_address,
street_number: this.getComponent(components, 'street_number'),
route: this.getComponent(components, 'route'),
city: this.getComponent(components, 'locality'),
state: this.getComponent(components, 'administrative_area_level_1'),
postal_code: this.getComponent(components, 'postal_code'),
country: this.getComponent(components, 'country'),
latitude: result.geometry.location.lat,
longitude: result.geometry.location.lng,
};
logger.info(`Address validated: ${parsed.formatted_address}`);
return {
success: true,
valid: true,
parsed_address: parsed,
};
} else {
return {
success: true,
valid: false,
error: 'Address not found',
};
}
} catch (error) {
logger.error(`Address validation failed: ${error.message}`);
throw error;
}
}
private getComponent(components: any[], type: string): string {
const component = components.find((c) => c.types.includes(type));
return component?.long_name || '';
}
}
Step 4: Create SMS Service Connector (Basic Auth)
Create src/connectors/sms_service.json:
{
"name": "sms_service",
"label": "SMS Service",
"description": "Twilio-compatible SMS delivery",
"type": "REST_API",
"config": {
"baseUrl": "https://api.twilio.com/2010-04-01",
"auth": {
"type": "BASIC_AUTH",
"username": "%%TWILIO_ACCOUNT_SID%%",
"password": "%%TWILIO_AUTH_TOKEN%%"
},
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"timeout": 15000
}
}
Environment Variables:
TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
TWILIO_AUTH_TOKEN=your_auth_token_here
Using SMS Connector
Create src/action-types/sendSMS.ts:
import { Record, String } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
import querystring from 'querystring';
const SendSMSParams = Record({
to: String,
message: String,
});
export class SendSMS extends S9AutomationActionType {
key = 'send_sms';
name = 'Send SMS';
async exec(params: any) {
const validated = SendSMSParams.check(params);
const { connectors, logger } = this.context;
try {
const smsService = connectors['sms_service'];
const accountSid = process.env.TWILIO_ACCOUNT_SID;
// Twilio uses form-encoded data
const body = querystring.stringify({
To: validated.to,
From: process.env.TWILIO_PHONE_NUMBER,
Body: validated.message,
});
const response = await smsService.call({
method: 'POST',
path: `/Accounts/${accountSid}/Messages.json`,
body: body,
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
},
});
logger.info(`SMS sent to ${validated.to}`);
return {
success: true,
message_sid: response.sid,
message: 'SMS sent successfully',
};
} catch (error) {
logger.error(`SMS sending failed: ${error.message}`);
throw error;
}
}
}
Step 5: Create CRM Connector (OAuth2)
Create src/connectors/crm_api.json:
{
"name": "crm_api",
"label": "CRM API",
"description": "Salesforce-compatible CRM integration",
"type": "REST_API",
"config": {
"baseUrl": "https://api.crm-system.com/v2",
"auth": {
"type": "OAUTH2",
"tokenUrl": "https://api.crm-system.com/oauth/token",
"clientId": "%%CRM_CLIENT_ID%%",
"clientSecret": "%%CRM_CLIENT_SECRET%%",
"scope": "contacts:write contacts:read"
},
"headers": {
"Content-Type": "application/json",
"Accept": "application/json"
},
"timeout": 30000
}
}
Environment Variables:
CRM_CLIENT_ID=your_client_id
CRM_CLIENT_SECRET=your_client_secret
Using CRM Connector
Create src/action-types/syncContactToCRM.ts:
import { Record, Number } from 'runtypes';
import { S9AutomationActionType } from '@april9/stack9-sdk';
export class SyncContactToCRM extends S9AutomationActionType {
key = 'sync_contact_to_crm';
name = 'Sync Contact to CRM';
async exec(params: any) {
const { customer_id } = params;
const { db, connectors, logger } = this.context;
try {
// Fetch customer
const customer = await db('customer')
.where({ id: customer_id })
.first();
if (!customer) {
throw new Error(`Customer ${customer_id} not found`);
}
const crmApi = connectors['crm_api'];
// Check if contact already exists in CRM
const existingContacts = await crmApi.call({
method: 'GET',
path: '/contacts',
params: {
email: customer.email,
},
});
if (existingContacts.data && existingContacts.data.length > 0) {
// Update existing contact
const contactId = existingContacts.data[0].id;
await crmApi.call({
method: 'PUT',
path: `/contacts/${contactId}`,
body: {
first_name: customer.first_name,
last_name: customer.last_name,
email: customer.email,
phone: customer.phone,
company: customer.company,
status: customer.status,
custom_fields: {
stack9_customer_id: customer.id,
last_synced: new Date().toISOString(),
},
},
});
logger.info(`Updated CRM contact ${contactId} for customer ${customer.email}`);
return {
success: true,
crm_contact_id: contactId,
action: 'updated',
};
} else {
// Create new contact
const newContact = await crmApi.call({
method: 'POST',
path: '/contacts',
body: {
first_name: customer.first_name,
last_name: customer.last_name,
email: customer.email,
phone: customer.phone,
company: customer.company,
status: customer.status,
custom_fields: {
stack9_customer_id: customer.id,
created_in_stack9: new Date().toISOString(),
},
},
});
logger.info(`Created CRM contact ${newContact.id} for customer ${customer.email}`);
return {
success: true,
crm_contact_id: newContact.id,
action: 'created',
};
}
} catch (error) {
logger.error(`CRM sync failed: ${error.message}`);
throw error;
}
}
}
Step 6: Error Handling Patterns
Pattern 1: Retry with Exponential Backoff
export class RetryableAction extends S9AutomationActionType {
key = 'retryable_action';
name = 'Retryable Action';
async exec(params: any) {
const maxRetries = 3;
let lastError: Error;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
const result = await this.makeApiCall(params);
return result;
} catch (error) {
lastError = error;
if (attempt < maxRetries) {
// Exponential backoff: 1s, 2s, 4s
const delay = Math.pow(2, attempt - 1) * 1000;
this.context.logger.warn(
`Attempt ${attempt} failed, retrying in ${delay}ms`
);
await this.sleep(delay);
}
}
}
throw new Error(`Failed after ${maxRetries} attempts: ${lastError.message}`);
}
private async makeApiCall(params: any) {
// Your API call here
}
private sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
}
Pattern 2: Circuit Breaker
class CircuitBreaker {
private failures = 0;
private lastFailureTime = 0;
private readonly threshold = 5;
private readonly timeout = 60000; // 1 minute
async call<T>(fn: () => Promise<T>): Promise<T> {
// Check if circuit is open
if (this.failures >= this.threshold) {
const timeSinceLastFailure = Date.now() - this.lastFailureTime;
if (timeSinceLastFailure < this.timeout) {
throw new Error('Circuit breaker is open - too many failures');
}
// Reset after timeout
this.failures = 0;
}
try {
const result = await fn();
this.failures = 0; // Reset on success
return result;
} catch (error) {
this.failures++;
this.lastFailureTime = Date.now();
throw error;
}
}
}
export class ResilientAction extends S9AutomationActionType {
private circuitBreaker = new CircuitBreaker();
async exec(params: any) {
return this.circuitBreaker.call(async () => {
const { connectors } = this.context;
return await connectors['external_api'].call({
method: 'GET',
path: '/data',
});
});
}
}
Pattern 3: Timeout Handling
async function withTimeout<T>(
promise: Promise<T>,
timeoutMs: number,
errorMessage = 'Operation timed out'
): Promise<T> {
let timeoutHandle: NodeJS.Timeout;
const timeoutPromise = new Promise<T>((_, reject) => {
timeoutHandle = setTimeout(() => {
reject(new Error(errorMessage));
}, timeoutMs);
});
try {
return await Promise.race([promise, timeoutPromise]);
} finally {
clearTimeout(timeoutHandle!);
}
}
export class TimeoutAction extends S9AutomationActionType {
async exec(params: any) {
const { connectors } = this.context;
try {
// 10 second timeout
const result = await withTimeout(
connectors['slow_api'].call({ method: 'GET', path: '/data' }),
10000,
'API call timed out after 10 seconds'
);
return { success: true, data: result };
} catch (error) {
if (error.message.includes('timed out')) {
this.context.logger.error('API timeout - using cached data');
// Fallback to cached data
return { success: true, cached: true, data: await this.getCachedData() };
}
throw error;
}
}
}
Pattern 4: Rate Limiting
class RateLimiter {
private requests: number[] = [];
private readonly limit: number;
private readonly windowMs: number;
constructor(limit: number, windowMs: number) {
this.limit = limit;
this.windowMs = windowMs;
}
async throttle(): Promise<void> {
const now = Date.now();
// Remove old requests outside the window
this.requests = this.requests.filter((time) => now - time < this.windowMs);
if (this.requests.length >= this.limit) {
const oldestRequest = this.requests[0];
const waitTime = this.windowMs - (now - oldestRequest);
await new Promise((resolve) => setTimeout(resolve, waitTime));
return this.throttle(); // Recursive retry
}
this.requests.push(now);
}
}
export class RateLimitedAction extends S9AutomationActionType {
// 100 requests per minute
private rateLimiter = new RateLimiter(100, 60000);
async exec(params: any) {
await this.rateLimiter.throttle();
const { connectors } = this.context;
return await connectors['api'].call({
method: 'GET',
path: '/resource',
});
}
}
Step 7: Testing External Integrations
Test Payment Processing
# Test payment action
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "process_payment",
"params": {
"amount": 99.99,
"currency": "USD",
"customer_email": "test@example.com",
"order_id": 1
}
}'
Test Email Sending
# Test email action
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "send_email",
"params": {
"to": "recipient@example.com",
"subject": "Test Email",
"template": "welcome",
"data": {
"first_name": "John"
}
}
}'
Test Address Validation
# Test address lookup
curl -X POST http://localhost:3000/api/test-action \
-H "Content-Type: application/json" \
-d '{
"actionType": "validate_address",
"params": {
"address": "1600 Amphitheatre Parkway, Mountain View, CA"
}
}'
Step 8: Advanced Patterns
Pattern 1: Webhook Signature Verification
import crypto from 'crypto';
export class VerifyWebhook extends S9AutomationActionType {
key = 'verify_webhook';
name = 'Verify Webhook Signature';
async exec(params: any) {
const { payload, signature, secret } = params;
// Calculate expected signature
const expectedSignature = crypto
.createHmac('sha256', secret)
.update(JSON.stringify(payload))
.digest('hex');
const isValid = crypto.timingSafeEqual(
Buffer.from(signature),
Buffer.from(expectedSignature)
);
if (!isValid) {
throw new Error('Invalid webhook signature');
}
return {
success: true,
valid: true,
message: 'Webhook signature verified',
};
}
}
Pattern 2: Pagination Handling
export class FetchAllRecords extends S9AutomationActionType {
async exec(params: any) {
const { connectors, logger } = this.context;
const api = connectors['external_api'];
let allRecords = [];
let page = 1;
let hasMore = true;
while (hasMore) {
logger.info(`Fetching page ${page}`);
const response = await api.call({
method: 'GET',
path: '/records',
params: {
page: page,
per_page: 100,
},
});
allRecords = allRecords.concat(response.data);
hasMore = response.has_more;
page++;
// Safety limit
if (page > 100) {
logger.warn('Reached maximum page limit (100)');
break;
}
}
logger.info(`Fetched total of ${allRecords.length} records`);
return {
success: true,
total_records: allRecords.length,
records: allRecords,
};
}
}
Pattern 3: Bulk API Operations
export class BulkSync extends S9AutomationActionType {
async exec(params: any) {
const { db, connectors, logger } = this.context;
// Fetch records that need syncing
const records = await db('customer')
.where('needs_sync', true)
.limit(1000);
logger.info(`Syncing ${records.length} customers`);
// Batch records (API allows max 100 per request)
const batchSize = 100;
const batches = [];
for (let i = 0; i < records.length; i += batchSize) {
batches.push(records.slice(i, i + batchSize));
}
const api = connectors['external_api'];
let successCount = 0;
let errorCount = 0;
for (const [index, batch] of batches.entries()) {
try {
logger.info(`Processing batch ${index + 1}/${batches.length}`);
await api.call({
method: 'POST',
path: '/bulk/customers',
body: {
customers: batch.map((c) => ({
email: c.email,
name: `${c.first_name} ${c.last_name}`,
status: c.status,
})),
},
});
// Mark as synced
await db('customer')
.whereIn('id', batch.map((c) => c.id))
.update({ needs_sync: false, last_synced: new Date() });
successCount += batch.length;
} catch (error) {
logger.error(`Batch ${index + 1} failed: ${error.message}`);
errorCount += batch.length;
}
}
return {
success: true,
synced: successCount,
failed: errorCount,
total: records.length,
};
}
}
Best Practices
Security
- Never hardcode credentials - Always use environment variables
- Validate webhook signatures - Verify requests from external services
- Use HTTPS only - Never send credentials over HTTP
- Rotate API keys regularly - Implement key rotation policies
- Limit API key permissions - Use least privilege principle
Error Handling
- Implement retries - Transient failures are common with APIs
- Use circuit breakers - Prevent cascading failures
- Set appropriate timeouts - Don't let operations hang forever
- Log all errors - Include request/response details
- Provide fallbacks - Gracefully degrade when APIs are down
Performance
- Cache responses - Cache frequently accessed data
- Use batch operations - Reduce number of API calls
- Implement rate limiting - Respect API rate limits
- Paginate large datasets - Don't fetch everything at once
- Use async operations - Don't block on long-running calls
Monitoring
- Track API usage - Monitor request counts and quotas
- Log response times - Identify slow APIs
- Alert on failures - Set up alerts for critical integrations
- Monitor rate limits - Don't hit rate limit ceilings
- Track costs - Many APIs charge per request
Troubleshooting
Authentication Failures
Problem: 401 Unauthorized responses
Solutions:
- Verify API key/token is correct
- Check environment variable is set
- Ensure auth header format matches API requirements
- Check if token has expired (OAuth2)
- Verify API key has required permissions
Timeout Errors
Problem: Requests timing out
Solutions:
- Increase timeout in connector config
- Check if external API is down (status page)
- Implement retry logic with exponential backoff
- Use async operations for long requests
- Contact API support if persistent
Rate Limit Exceeded
Problem: 429 Too Many Requests
Solutions:
- Implement rate limiting in your code
- Add delays between requests
- Use batch endpoints when available
- Cache responses to reduce calls
- Upgrade API plan if needed
Invalid Response Format
Problem: Unexpected API response structure
Solutions:
- Check API documentation for response format
- Log full response for debugging
- Handle different response formats
- Check API version hasn't changed
- Validate response schema
SSL/TLS Errors
Problem: Certificate verification failures
Solutions:
- Ensure correct base URL (https://)
- Check server certificate is valid
- Update Node.js version if outdated
- For testing only: disable cert verification (not production!)
- Contact API provider about certificate issues
Next Steps
You've mastered external API integration! Continue learning:
- Entity Hooks Reference - Advanced hook patterns
- Action Types Reference - Master action types
- Connectors Reference - All connector types
- Automations Reference - Complete automation guide