Mimir AIP Wiki

Comprehensive documentation and guides for using the Mimir AIP platform.

View on GitHub Quick Start

Contents

Quick Links

New to Mimir AIP?

This documentation will help you get up and running quickly with the platform.

Mimir AIP is designed to be modular and extensible. It allows you to define complex workflows with minimal coding, focusing instead on configuration and pipeline design.

Project Overview

Mimir-AIP is a modular, plugin-driven pipeline framework for automating data processing, AI/LLM tasks, and report generation. The platform is designed with these key principles:

Extensibility

Custom plugins allow you to integrate with any API, data source, or processing step.

Robust Error Handling

Comprehensive logging and error management to prevent silent failures.

Flexible Testing

Test mode with mock responses allows for rapid development and validation.

YAML-Driven

Define complex workflows without writing code, just using YAML configuration.

Getting Started

Installation
  1. Clone the repository:
    git clone https://github.com/Mimir-AIP/Mimir-AIP.git
  2. Set up a Python virtual environment (recommended):
    python -m venv venv
    source venv/bin/activate # On Windows: venv\Scripts\activate
  3. Install dependencies:
    pip install -r requirements.txt
Project Structure
Mimir-AIP/
├── src/
│   ├── Plugins/           # All plugin code (AIModels, Input, Output, Data_Processing, etc.)
│   ├── pipelines/         # Pipeline YAML definitions
│   ├── config.yaml        # Main configuration file
│   └── main.py            # Pipeline runner
├── tests/                 # Unit and integration tests
├── requirements.txt       # Python dependencies
├── Documentation.md       # (This file)
└── ...

Pipelines

Pipelines are the heart of Mimir AIP. They define the sequence of operations to be performed on your data.

Example Pipeline Step
- name: "Generate Section Summary"
  plugin: "LLMFunction"
  config:
    prompt: "Summarize this section: {section_text}"
    model: "openrouter/mistral-7b"
    mock_response: {"section_summary": "This is a mock section summary for testing."}  # For test mode
  output: "section_summary"
Pipeline Configuration

Reference your pipeline in src/config.yaml:

pipelines:
  - name: "BBC News Pipeline"
    file: "pipelines/POC.yaml"
    enabled: true
Pipeline Variables

Outputs from each step are available to subsequent steps using the {variable_name} syntax in your configuration. This enables a modular data flow between steps.

API Keys

Mimir AIP relies on external APIs for many capabilities. Here's how to set up your API keys securely:

Environment Variables

API keys are loaded via environment variables for security. For OpenRouter, set OPENROUTER_API_KEY in your environment:

On Mac/Linux:

export OPENROUTER_API_KEY=your_key_here

On Windows:

set OPENROUTER_API_KEY=your_key_here
Important Note

The OpenRouter plugin will raise an error if the required API key is missing. Always ensure your environment variables are properly set before running pipelines.

Plugins

Plugins are the building blocks of Mimir AIP. They handle specific tasks within your pipeline.

Plugin Structure
  • Plugins live under src/Plugins/<Type>/<PluginName>/<PluginName>.py.
  • Types include: AIModels, Input, Output, Data_Processing, etc.
Creating a Plugin

Each plugin must subclass BasePlugin and implement the execute_pipeline_step method:

from Plugins.BasePlugin import BasePlugin

class MyPlugin(BasePlugin):
    plugin_type = "Data_Processing"
    def execute_pipeline_step(self, step_config, context):
        # Your logic here
        return {step_config["output"]: result}

Test Mode

Test mode allows you to develop and test pipelines without making real API calls.

Enabling Test Mode

Enable test mode in src/config.yaml:

settings:
  test_mode: true
Test Mode Features
  • In test mode, LLMFunction and similar plugins will use mock_response from the pipeline YAML instead of making API calls.
  • All steps requiring external APIs should have explicit mock_response fields for predictable tests.
  • Output files are automatically cleaned up at the start of each test-mode run to ensure a clean testing environment.

FAQ

Plugins are auto-discovered based on their location and naming conventions. Simply place your plugin in the correct directory structure and it will be available in your pipelines.

Environment variables are preferred for secrets/API keys. Never hardcode them in your pipeline files or source code. Use a .env file for local development if needed.

Test mode is strict: all LLM/API steps must have explicit mock responses defined in their configuration. The system will use these mocks instead of making real API calls.

Output directories may be relative to src/—use absolute paths if needed for specific locations. Check your pipeline configuration for the output path settings.

Logging is your friend! Check mimir.log for detailed information about pipeline execution, errors, and warnings. Enable debug logging for more verbose output.