API Development and REST

Master modern API design, development, and integration patterns for building scalable web services

What You'll Learn

Introduction to APIs

An Application Programming Interface (API) is a set of rules and protocols that allows different software applications to communicate with each other. APIs are the backbone of modern software architecture, enabling integration between systems, services, and platforms.

What is an API?

Think of an API as a waiter in a restaurant. You (the client) look at the menu (API documentation), order your meal (make a request), the waiter takes your order to the kitchen (server), and returns with your food (response). You don't need to know how the kitchen works internally.

Types of APIs

REST (Representational State Transfer)

Most popular, uses HTTP methods, stateless, resource-based

Use Case: Web services, CRUD operations, microservices

GraphQL

Query language for APIs, client specifies exactly what data it needs

Use Case: Complex data requirements, mobile apps

SOAP (Simple Object Access Protocol)

XML-based protocol, strict standards, enterprise-focused

Use Case: Financial services, legacy systems

gRPC

High-performance RPC framework using Protocol Buffers

Use Case: Microservices, real-time communication

WebSocket

Full-duplex communication, persistent connection

Use Case: Real-time apps, chat, gaming

Webhooks

Event-driven, server pushes data to client

Use Case: Notifications, event triggers

Industry Trend: REST APIs account for over 80% of public APIs, but GraphQL adoption is growing rapidly, especially for mobile and complex applications.

REST Architecture

REST (Representational State Transfer) is an architectural style for designing networked applications. It relies on stateless, client-server communication using HTTP.

REST Principles (Constraints)

1. Client-Server Architecture

Separation of concerns: UI and data storage are separated, allowing independent evolution.

2. Statelessness

Each request contains all information needed to process it. Server doesn't store client context between requests.

3. Cacheability

Responses must define themselves as cacheable or non-cacheable to improve performance.

4. Uniform Interface

Standard way to communicate between components:

5. Layered System

Client cannot tell if connected directly to end server or intermediary (load balancer, cache, etc.)

6. Code on Demand (Optional)

Servers can extend client functionality by transferring executable code (JavaScript, etc.)

REST Resource Design

Resource-Based URL Structure:

✅ GOOD (Noun-based, hierarchical):
GET    /api/v1/users                 # Get all users
GET    /api/v1/users/123             # Get user by ID
POST   /api/v1/users                 # Create new user
PUT    /api/v1/users/123             # Update user (full)
PATCH  /api/v1/users/123             # Update user (partial)
DELETE /api/v1/users/123             # Delete user
GET    /api/v1/users/123/orders      # Get orders for user
GET    /api/v1/users/123/orders/456  # Get specific order

❌ BAD (Verb-based, inconsistent):
GET    /api/getUsers
POST   /api/createUser
GET    /api/user/get?id=123
POST   /api/deleteUser?id=123

Richardson Maturity Model

Level 0: The Swamp of POX (Plain Old XML)
- Single URI, single HTTP method (usually POST)
- Example: Traditional SOAP/XML-RPC

Level 1: Resources
- Multiple URIs, single HTTP method
- Each resource has its own URI

Level 2: HTTP Verbs
- Multiple URIs, multiple HTTP methods
- Proper use of GET, POST, PUT, DELETE, etc.
- Appropriate status codes (200, 404, 500)

Level 3: Hypermedia Controls (HATEOAS)
- Responses include links to related resources
- Client discovers API capabilities dynamically

Most REST APIs operate at Level 2.

HTTP Methods and Status Codes

HTTP Methods (Verbs)

Method Purpose Idempotent Safe Request Body Response Body
GET Retrieve resource Yes Yes No Yes
POST Create resource No No Yes Yes
PUT Update/Replace resource Yes No Yes Yes
PATCH Partial update No No Yes Yes
DELETE Delete resource Yes No No Optional
HEAD Get metadata only Yes Yes No No
OPTIONS Get allowed methods Yes Yes No Yes

HTTP Status Codes

2xx Success

  • 200 OK: Request succeeded
  • 201 Created: Resource created
  • 202 Accepted: Request accepted, processing
  • 204 No Content: Success, no response body

3xx Redirection

  • 301 Moved Permanently: Resource moved
  • 302 Found: Temporary redirect
  • 304 Not Modified: Cached version is current

4xx Client Errors

  • 400 Bad Request: Invalid syntax
  • 401 Unauthorized: Authentication required
  • 403 Forbidden: Access denied
  • 404 Not Found: Resource doesn't exist
  • 409 Conflict: Request conflicts with current state
  • 422 Unprocessable Entity: Validation errors
  • 429 Too Many Requests: Rate limit exceeded

5xx Server Errors

  • 500 Internal Server Error: Generic server error
  • 502 Bad Gateway: Invalid response from upstream
  • 503 Service Unavailable: Server overloaded/down
  • 504 Gateway Timeout: Upstream timeout

Example: Complete CRUD Operations

// Create User
POST /api/v1/users
Content-Type: application/json

{
  "name": "John Doe",
  "email": "john@example.com",
  "role": "user"
}

Response: 201 Created
{
  "id": 123,
  "name": "John Doe",
  "email": "john@example.com",
  "role": "user",
  "createdAt": "2024-01-15T10:30:00Z"
}

// Read User
GET /api/v1/users/123

Response: 200 OK
{
  "id": 123,
  "name": "John Doe",
  "email": "john@example.com",
  "role": "user",
  "createdAt": "2024-01-15T10:30:00Z"
}

// Update User (Full Replace)
PUT /api/v1/users/123
Content-Type: application/json

{
  "name": "John Smith",
  "email": "john.smith@example.com",
  "role": "admin"
}

Response: 200 OK
{
  "id": 123,
  "name": "John Smith",
  "email": "john.smith@example.com",
  "role": "admin",
  "updatedAt": "2024-01-15T11:00:00Z"
}

// Partial Update
PATCH /api/v1/users/123
Content-Type: application/json

{
  "role": "admin"
}

Response: 200 OK
{
  "id": 123,
  "name": "John Smith",
  "email": "john.smith@example.com",
  "role": "admin",
  "updatedAt": "2024-01-15T11:05:00Z"
}

// Delete User
DELETE /api/v1/users/123

Response: 204 No Content

API Design Principles

1. Use Nouns for Resources, Not Verbs

✅ GOOD:
GET    /users
POST   /users
GET    /users/123

❌ BAD:
GET    /getUsers
POST   /createUser
GET    /getUser?id=123

2. Pluralize Resource Names

✅ GOOD:
/users
/products
/orders

❌ BAD:
/user
/product
/order

3. Use Nested Resources for Relationships

✅ GOOD:
GET /users/123/orders           # Get all orders for user 123
GET /users/123/orders/456       # Get specific order for user 123
POST /users/123/orders          # Create order for user 123

⚠️ Avoid deep nesting (max 2-3 levels):
/users/123/orders/456/items/789/reviews  # Too deep!

4. Use Query Parameters for Filtering, Sorting, Pagination

// Filtering
GET /users?role=admin&status=active

// Sorting
GET /users?sort=created_at&order=desc

// Pagination
GET /users?page=2&limit=20
GET /users?offset=20&limit=20

// Searching
GET /users?search=john

// Field Selection
GET /users?fields=id,name,email

// Combined
GET /users?role=admin&sort=created_at&page=1&limit=10

5. Consistent Naming Conventions

Choose one style and stick to it:

✅ snake_case (recommended for JSON):
{
  "user_id": 123,
  "first_name": "John",
  "created_at": "2024-01-15"
}

✅ camelCase (common in JavaScript):
{
  "userId": 123,
  "firstName": "John",
  "createdAt": "2024-01-15"
}

❌ Inconsistent:
{
  "user_id": 123,
  "firstName": "John",
  "created_at": "2024-01-15"
}

6. Return Consistent Response Structures

// Single Resource
{
  "data": {
    "id": 123,
    "name": "John Doe"
  }
}

// Collection
{
  "data": [
    {"id": 123, "name": "John Doe"},
    {"id": 124, "name": "Jane Smith"}
  ],
  "meta": {
    "page": 1,
    "per_page": 20,
    "total": 150,
    "total_pages": 8
  },
  "links": {
    "self": "/users?page=1",
    "next": "/users?page=2",
    "last": "/users?page=8"
  }
}

// Error Response
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid input data",
    "details": [
      {
        "field": "email",
        "message": "Email is required"
      },
      {
        "field": "age",
        "message": "Age must be at least 18"
      }
    ]
  }
}

Authentication and Authorization

Authentication Methods

1. API Keys

Use Case: Simple authentication for server-to-server communication

GET /api/v1/users
X-API-Key: your-api-key-here

Pros:
+ Simple to implement
+ Good for public APIs

Cons:
- Less secure (keys can't expire easily)
- No user context
- Difficult to rotate

2. OAuth 2.0

Use Case: Third-party authorization, social login

OAuth 2.0 Flow (Authorization Code):

1. Client requests authorization from user
2. User grants permission
3. Client receives authorization code
4. Client exchanges code for access token
5. Client uses access token to access API

GET /api/v1/users
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Pros:
+ Industry standard
+ Secure token-based
+ Supports multiple grant types
+ Tokens can expire

Cons:
- Complex to implement
- Requires token management

3. JWT (JSON Web Tokens)

Use Case: Stateless authentication, microservices

JWT Structure: header.payload.signature

// Header
{
  "alg": "HS256",
  "typ": "JWT"
}

// Payload
{
  "sub": "123",
  "name": "John Doe",
  "role": "admin",
  "iat": 1516239022,
  "exp": 1516242622
}

// Usage
GET /api/v1/users
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

Pros:
+ Self-contained (no database lookup)
+ Stateless
+ Cross-domain/microservices friendly
+ Can include claims

Cons:
- Cannot be revoked easily
- Size (larger than session IDs)
- Security risk if not implemented correctly

4. Basic Authentication

Use Case: Simple internal APIs, development

GET /api/v1/users
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

// Base64 encoded: "username:password"

⚠️ MUST use HTTPS in production

Pros:
+ Very simple
+ Built into HTTP

Cons:
- Credentials sent with every request
- No expiration
- Less secure

Authorization Patterns

Role-Based Access Control (RBAC)

Users assigned roles, roles have permissions

Roles: Admin, User, Guest
Admin can: CRUD all resources
User can: Read, Create own
Guest can: Read only

Attribute-Based Access Control (ABAC)

Access based on attributes (user, resource, environment)

Allow if:
- User.role = "manager"
- Resource.department = User.department
- Time = business_hours

Permission-Based

Fine-grained permissions per user

User permissions:
- users.read
- users.create
- posts.edit.own
- posts.delete.all

Scope-Based (OAuth)

Token includes scopes defining access

Token scopes:
read:users
write:posts
delete:comments

API Versioning

API versioning allows you to make changes without breaking existing clients. There are several strategies:

1. URI Versioning (Most Common)

https://api.example.com/v1/users
https://api.example.com/v2/users

Pros:
+ Simple and clear
+ Easy to route
+ Visible in URLs

Cons:
- Pollutes URI space
- Multiple versions = multiple endpoints

2. Header Versioning

GET /users
Accept-Version: v2
# or
API-Version: 2

Pros:
+ Clean URIs
+ RESTful

Cons:
- Less visible
- Harder to test in browser

3. Content Negotiation (Accept Header)

GET /users
Accept: application/vnd.example.v2+json

Pros:
+ RESTful
+ Follows HTTP standards

Cons:
- Complex
- Harder to debug

4. Query Parameter Versioning

https://api.example.com/users?version=2

Pros:
+ Easy to implement

Cons:
- Can conflict with other parameters
- Less RESTful

Versioning Best Practices

Error Handling

Consistent Error Response Structure

// Standard Error Format
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid request data",
    "details": [
      {
        "field": "email",
        "message": "Email is required",
        "code": "REQUIRED_FIELD"
      },
      {
        "field": "age",
        "message": "Age must be at least 18",
        "code": "MIN_VALUE",
        "constraint": 18
      }
    ],
    "timestamp": "2024-01-15T10:30:00Z",
    "path": "/api/v1/users",
    "requestId": "abc-123-def-456"
  }
}

// Simple Error (4xx)
{
  "error": {
    "code": "NOT_FOUND",
    "message": "User with ID 123 not found"
  }
}

// Server Error (5xx)
{
  "error": {
    "code": "INTERNAL_ERROR",
    "message": "An unexpected error occurred",
    "requestId": "xyz-789"
  }
}

Common Error Codes

Client Errors (4xx)

  • VALIDATION_ERROR (400): Invalid input
  • UNAUTHORIZED (401): Missing/invalid auth
  • FORBIDDEN (403): Insufficient permissions
  • NOT_FOUND (404): Resource doesn't exist
  • CONFLICT (409): Resource already exists
  • RATE_LIMIT_EXCEEDED (429): Too many requests

Server Errors (5xx)

  • INTERNAL_ERROR (500): Generic server error
  • SERVICE_UNAVAILABLE (503): Maintenance/overload
  • GATEWAY_TIMEOUT (504): Upstream timeout

Error Handling Examples

// Example: Node.js/Express Error Handler
app.use((err, req, res, next) => {
  // Log error for debugging
  console.error(err);

  // Validation errors
  if (err.name === 'ValidationError') {
    return res.status(400).json({
      error: {
        code: 'VALIDATION_ERROR',
        message: err.message,
        details: err.errors
      }
    });
  }

  // Not found errors
  if (err.name === 'NotFoundError') {
    return res.status(404).json({
      error: {
        code: 'NOT_FOUND',
        message: err.message
      }
    });
  }

  // Unauthorized
  if (err.name === 'UnauthorizedError') {
    return res.status(401).json({
      error: {
        code: 'UNAUTHORIZED',
        message: 'Authentication required'
      }
    });
  }

  // Generic server error
  res.status(500).json({
    error: {
      code: 'INTERNAL_ERROR',
      message: 'An unexpected error occurred',
      requestId: req.id
    }
  });
});

Security Best Practices

Essential Security Measures

1. HTTPS Only

Always use HTTPS in production to encrypt data in transit

// Redirect HTTP to HTTPS
if (req.protocol !== 'https') {
  res.redirect('https://' + req.hostname + req.url);
}

2. Input Validation

Validate all input to prevent injection attacks

// Validate and sanitize
const schema = {
  email: Joi.string().email(),
  age: Joi.number().min(0).max(150)
};

validate(req.body, schema);

3. Rate Limiting

Prevent abuse and DDoS attacks

// Rate limit: 100 requests per 15 min
const limiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 100
});

app.use('/api', limiter);

4. CORS Configuration

Control cross-origin access

// Configure CORS
app.use(cors({
  origin: 'https://example.com',
  methods: ['GET', 'POST'],
  credentials: true
}));

5. API Keys Security

  • Never expose keys in client code
  • Use environment variables
  • Rotate keys regularly
  • Use different keys per environment

6. SQL Injection Prevention

Use parameterized queries

// ❌ BAD (vulnerable)
query(`SELECT * FROM users WHERE id = ${id}`);

// ✅ GOOD (safe)
query('SELECT * FROM users WHERE id = ?', [id]);

Security Headers

// Essential security headers
app.use(helmet({
  contentSecurityPolicy: true,
  hsts: {
    maxAge: 31536000,
    includeSubDomains: true
  },
  frameguard: {
    action: 'deny'
  },
  noSniff: true,
  xssFilter: true
}));

// Headers sent:
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block

OWASP API Security Top 10

API Documentation

Documentation Standards

OpenAPI Specification (Swagger)

Industry standard for documenting REST APIs

openapi: 3.0.0
info:
  title: User API
  version: 1.0.0
  description: API for managing users
paths:
  /users:
    get:
      summary: Get all users
      parameters:
        - name: page
          in: query
          schema:
            type: integer
            default: 1
        - name: limit
          in: query
          schema:
            type: integer
            default: 20
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/User'
    post:
      summary: Create a new user
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/UserInput'
      responses:
        '201':
          description: User created
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
components:
  schemas:
    User:
      type: object
      properties:
        id:
          type: integer
        name:
          type: string
        email:
          type: string
          format: email
        createdAt:
          type: string
          format: date-time

What to Document

Popular Documentation Tools

Swagger UI

Interactive documentation from OpenAPI spec

Features: Try it out, code generation

Postman

API development and documentation

Features: Collections, testing, mock servers

Redoc

Beautiful OpenAPI documentation

Features: Responsive, customizable themes

API Blueprint

Markdown-based API documentation

Features: Simple syntax, readable

Testing APIs

Types of API Tests

1. Unit Tests

Test individual API functions/handlers in isolation

// Jest example
describe('User API', () => {
  test('GET /users returns user list', async () => {
    const response = await request(app)
      .get('/api/v1/users')
      .expect(200);

    expect(response.body.data).toBeInstanceOf(Array);
    expect(response.body.data.length).toBeGreaterThan(0);
  });

  test('POST /users creates new user', async () => {
    const newUser = {
      name: 'John Doe',
      email: 'john@example.com'
    };

    const response = await request(app)
      .post('/api/v1/users')
      .send(newUser)
      .expect(201);

    expect(response.body.data).toHaveProperty('id');
    expect(response.body.data.name).toBe(newUser.name);
  });

  test('GET /users/999 returns 404', async () => {
    await request(app)
      .get('/api/v1/users/999')
      .expect(404);
  });
});

2. Integration Tests

Test API endpoints with real database and dependencies

describe('User Integration Tests', () => {
  beforeAll(async () => {
    await db.connect();
  });

  afterAll(async () => {
    await db.disconnect();
  });

  beforeEach(async () => {
    await db.clearUsers();
  });

  test('Complete user lifecycle', async () => {
    // Create user
    const createResponse = await request(app)
      .post('/api/v1/users')
      .send({ name: 'John', email: 'john@test.com' })
      .expect(201);

    const userId = createResponse.body.data.id;

    // Retrieve user
    const getResponse = await request(app)
      .get(`/api/v1/users/${userId}`)
      .expect(200);

    expect(getResponse.body.data.name).toBe('John');

    // Update user
    await request(app)
      .patch(`/api/v1/users/${userId}`)
      .send({ name: 'John Updated' })
      .expect(200);

    // Delete user
    await request(app)
      .delete(`/api/v1/users/${userId}`)
      .expect(204);

    // Verify deletion
    await request(app)
      .get(`/api/v1/users/${userId}`)
      .expect(404);
  });
});

3. Contract Tests

Verify API meets contract/specification

4. Performance Tests

Test API under load and stress conditions

Testing Tools

GraphQL vs REST

GraphQL Overview

GraphQL is a query language for APIs that allows clients to request exactly the data they need.

// GraphQL Query
query {
  user(id: 123) {
    id
    name
    email
    posts {
      id
      title
      comments {
        id
        text
      }
    }
  }
}

// Response (only requested fields)
{
  "data": {
    "user": {
      "id": 123,
      "name": "John Doe",
      "email": "john@example.com",
      "posts": [
        {
          "id": 1,
          "title": "First Post",
          "comments": [...]
        }
      ]
    }
  }
}

REST vs GraphQL Comparison

Aspect REST GraphQL
Data Fetching Multiple endpoints, fixed responses Single endpoint, flexible queries
Over-fetching Common (get unnecessary data) No (request only what you need)
Under-fetching Common (multiple requests needed) Rare (single query for nested data)
Versioning Explicit versions (v1, v2) No versioning needed (evolve schema)
Caching HTTP caching (easier) More complex (requires custom logic)
Learning Curve Lower (familiar HTTP concepts) Higher (new query language)
Tooling Mature, widespread Growing, less mature
Best For Simple CRUD, public APIs, caching needs Complex data, mobile apps, rapid iteration

When to Choose GraphQL

When to Choose REST

Performance Optimization

Caching Strategies

HTTP Caching

// Cache-Control headers
Cache-Control: public, max-age=3600
ETag: "33a64df551425fcc55e"

// Client sends:
If-None-Match: "33a64df551425fcc55e"

// Server responds:
304 Not Modified

Redis Caching

// Cache frequently accessed data
const cachedUser = await redis.get(`user:${id}`);
if (cachedUser) {
  return JSON.parse(cachedUser);
}

const user = await db.getUser(id);
await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 3600);
return user;

CDN Caching

Use CDN for static API responses and assets

  • Cloudflare
  • AWS CloudFront
  • Fastly

Database Query Caching

// Cache query results
const cacheKey = 'users:active';
let users = cache.get(cacheKey);

if (!users) {
  users = await db.query('SELECT * FROM users WHERE active = true');
  cache.set(cacheKey, users, 300);
}
return users;

Pagination

// Offset-based pagination (simple but slower for large offsets)
GET /users?page=2&limit=20
// Skip 20 records, return next 20

// Cursor-based pagination (better performance)
GET /users?cursor=eyJpZCI6MjB9&limit=20
// Start from cursor, return next 20

// Response includes next cursor
{
  "data": [...],
  "pagination": {
    "nextCursor": "eyJpZCI6NDB9",
    "hasMore": true
  }
}

Compression

// Enable gzip compression
app.use(compression({
  level: 6,
  threshold: 1024,
  filter: (req, res) => {
    if (req.headers['x-no-compression']) {
      return false;
    }
    return compression.filter(req, res);
  }
}));

// Headers
Accept-Encoding: gzip, deflate
Content-Encoding: gzip

Database Optimization

Async Processing

// For long-running operations, return immediately and process async
POST /api/v1/reports
Response: 202 Accepted
{
  "jobId": "abc123",
  "status": "processing",
  "statusUrl": "/api/v1/jobs/abc123"
}

// Client polls status
GET /api/v1/jobs/abc123
Response: 200 OK
{
  "jobId": "abc123",
  "status": "completed",
  "resultUrl": "/api/v1/reports/abc123"
}

Best Practices Summary

Design Best Practices

Security Best Practices

Performance Best Practices

Documentation Best Practices

Tools and Frameworks

Backend Frameworks

API Development Tools

Testing Tools

API Management

Monitoring and Analytics

Additional Resources

Related Topics

Recommended Reading

Online Resources

API Standards and Best Practice Guides

← Software Engineering IT Home Microservices →