Patterns of Application Development Using AI

About my latest book, now available on Leanpub

5 min readApr 7, 2024

Many of you keep up with what I’m working on via this blog, so I figured I’d tell you about my latest book. I just published it a week ago and it’s already at the top of the charts on Leanpub.

What makes this book different

In contrast to many other books out there about AI, this book is specifically about how to incorporate the use of AI-powered components in your otherwise totally normal software projects. It answers the question on the minds of millions of application developers around the world: How do I get started using AI?

The book is full of practical approaches, techniques, and patterns based on my real-life experience building Olympia, the world’s premier AI-powered consultant platform. I also include a lot of working code examples, that you can lift almost directly and plop right into your own projects.

Those code examples are written in Ruby, not Python or Typescript. The expressiveness of Ruby is a perfect match for AI, since it is based on natural language processing. I see what passes for good AI code examples in other languages and I cringe.

Introduction
Thoughts on Software Architecture
What is a Large Language Model?
Thinking About Performance
Experimenting With Different LLM Models

Part 1: Fundamental Approaches & Techniques

Narrow The Path
How The Path Gets “Narrowed”
Prompt Engineering
Retrieval Augmented Generation (RAG)
Prompt Distillation
Raw Versus Instruct-Tuned Models
Separating Instructions from Data: A Crucial Principle

Multitude of Workers
AI Workers As Independent Reusable Components
Account Management
E-commerce Applications
Healthcare Applications
AI Worker as a Process Manager
Integrating AI Workers Into Your Application Architecture
Composability and Orchestration of AI Workers

Using Tools
What is Tool Use?
The Potential of Tool Use
The Tool Use Workflow
Best Practices for Tool Use
Composing and Chaining Tools
Future Directions

Self Healing Data
Key Benefits
How It Works
Practical Case Study: Fixing Broken JSON

Contextual Content Generation
Key Benefits
Implementation
Generating Copy for User Interfaces
The Importance of User Testing and Feedback

Intelligent Workflow Orchestration
Business Need
Key Benefits
Key Patterns
Implementing Intelligent Workflow Orchestration in Practice
Monitoring and Logging
Scalability and Performance Considerations
Testing and Validation

Generative UI: Personalized Experiences At Scale
Defining Generative UI
Example
The Shift to Outcome-Oriented Design
Challenges and Considerations
Future Outlook and Opportunities

Part 2: The Patterns

Prompt Engineering Patterns
Prompt Object
Prompt Distillation
Prompt Template
Prompt Chaining
Feedback Loop
Query Analyzer
Query Rewriter
Ventriloquist

Discrete Components
Predicate
Function Dispatcher
API Bridge
Result Interpreter
Virtual Machine
Specification and Testing

Intelligent Error Handling
Contextual Error Diagnosis
Intelligent Error Reporting
Predictive Error Prevention
Self-Healing Error Recovery
Personalized Error Communication
Adaptive Error Handling Workflow

Human In The Loop (HITL)
Escalation Pattern
Passive Information Radiation Pattern
Collaborative Decision-Making Pattern
Continuous Learning Pattern

Pattern Example: Ventriloquist

This is an excerpt from Patterns of Application Development Using AI

The Ventriloquist pattern is a powerful technique in prompt engineering that allows you to guide the AI’s responses by preloading hardcoded user-assistant exchanges into the conversation transcript before starting any completions. This pattern enables you to narrow the path effectively and ensure that the AI stays on course, providing responses that align with your desired direction.

While the Ventriloquist pattern can be used to maintain a consistent voice, tone, or persona across the AI’s responses, its most significant benefit lies in its ability to subtly influence the AI’s behavior. By carefully crafting the initial exchanges and setting the context, you can steer the AI towards generating responses that meet your specific requirements.

How It Works

The Ventriloquist pattern involves the following steps:

Define the desired direction or outcome for the AI’s responses.
Create a set of hardcoded user-assistant exchanges that establish the context and guide the AI towards the intended direction.
Preload these exchanges into the conversation transcript before starting
any completions.
Initiate the chat completion process, allowing the AI to generate responses based on the preloaded context.
The AI, influenced by the initial exchanges, generates responses that align with the desired direction.

By preloading the conversation transcript with carefully crafted exchanges, you essentially put words in the AI’s mouth, much like a ventriloquist controlling their puppet. The AI, being a language model trained to maintain coherence and consistency, will naturally follow the established context and generate responses that continue along the same path.

When to Use It

The Ventriloquist pattern is particularly useful in scenarios where:

You want to guide the AI’s responses towards a specific topic, sentiment, or style.
You need to establish a particular context or background information before allowing the AI to generate responses.
You want to maintain consistency in the AI’s behavior across multiple conversations or sessions.
You need to provide examples or demonstrations of desired responses to influence the AI’s output.

By employing the Ventriloquist pattern, you can effectively control the direction of the conversation and ensure that the AI generates responses that meet your specific requirements.

Example

Here’s an example implementation of the Ventriloquist pattern in Ruby:

class AlternateKeywords
  include ChatCompletion
  
  PROMPT = <<~END
    Matching the original language of the question generate 3 alternate
    keywords that might produce better results. Reply with just the list,
    one per line.
  END

  def call(question)
    transcript << { user: question }
    transcript << { assistant: "Searching... no results found." }
    transcript << { user: PROMPT }
    chat_completion
  end

  def max_tokens
    30
  end

  def model
    [
      "databricks/dbrx-instruct:nitro",
      "cohere/command-r",
    ]
  end

  def temperature
    0.0
  end
end

In this simple example, the AlternateKeywords class demonstrates the Ventriloquist pattern by preloading a conversation transcript with specific user-assistant exchanges before generating alternate keywords.

The call method takes a question parameter and performs the following steps:

Appends the user’s question to the conversation transcript.
Appends a hardcoded assistant response indicating that no results were found.
Appends the PROMPT to the transcript, asking the AI to generate alternate keywords.
Initiates the chat completion process.

By preloading the transcript with the hardcoded “no results found” response, the AI is primed to understand that we are trying to execute a search. The final user prompt then explicitly asks for alternate keywords.

This pattern allows you to guide the AI’s behavior and obtain desired responses by carefully crafting the initial conversation transcript, and is sometimes more effective and succint than trying to describe your desired outcome using the prompt only.

I’ll continue updating this blog post with additional notes and excerpts over time. Last updated: Sunday, April 7th, 2024.

All buyers of Patterns of Application Development Using AI at Leanpub will get a coupon code for the physical book when it’s published in a few months.