Project files

MindPaw AI Infra: Researcher Path

This directory is an optional AI infrastructure experimental field. Beginners do not need to start it; simply follow MindPaw_main/README.md to reproduce the Demo.

The research goal is to explore edge-cloud Agent runtime on resource-constrained robots: selecting the execution location among local, Gateway, and cloud based on task latency, risk, power, network, and privacy requirements.

ESP8266 robot -> Edge Gateway -> local model / cloud LLM
       └──────── direct cloud mode remains available ────────┘

Current MVP Capabilities

• OpenAI-compatible POST /v1/chat/completions
• Doubao/Fangzhou, OpenAI-compatible, local Ollama, and mock upstream
• Gateway token authentication
• Upstream timeout and error conversion
• Boundary validation for MindPaw actions, expressions, and melody JSON schemas
• Docker and local Python startup
• 2.0 Streaming 3D Reconstruction Perception Layer (recon/) — End-to-end ~150 ms obstacle avoidance latency; see recon/README.md and Docs/09_Streaming_Recon.md

The current Gateway serves as a research starting point and does not claim to have a complete scheduling system. Future extensions should focus on measurable system issues rather than simply increasing the number of Providers.

Research Questions

It is recommended to structure your paper or experimental report around the following questions:

For robots with limited resources, how can we dynamically select the execution location of the Agent—local device, edge gateway, or cloud—based on task SLO (Service Level Objective), energy consumption, network conditions, privacy, and action risk?

Recommended three-tier execution model:

The strategy can be formulated as a multi-objective optimization:

latency + energy + cloud_cost + privacy_exposure + action_risk

Suggested Experimental Route

Phase 1: Establishing a Baseline

Compare the following strategies:

1. ESP8266 directly accessing the cloud.
2. All tasks routed through the Gateway.
3. Local/cloud switching based on fixed rules.
4. Switching based solely on network latency.
5. Adaptive strategy.

Record real voice, gesture, web chat, and motion tasks—do not rely solely on synthetic requests.

Phase 2: Incorporating Runtime Mechanisms

• Asynchronous task queue and request_id.
• Device/session isolation.
• Network status and power-aware routing.
• Local fallback and offline degradation.
• Action whitelist and expiration time.
• Semantic caching and duplicate request deduplication.
• Request duration, cost, token usage, and failure rate statistics.

Phase 3: Establishing Reproducible Experiments

At minimum, report the following:

• P50/P95/P99 end-to-end latency.
• Action success rate and JSON schema failure rate.
• Network recovery time after disconnection.
• Power consumption of ESP8266, Gateway, and cloud.
• Token consumption and cost per task.
• Performance under different packet loss rates, RTT, and concurrency levels.

It is recommended to retain the experiments/ directory, which should include configurations, network traces, raw logs, plotting scripts, and result summaries. Paper results must be reproducible using Docker and fixed random seeds.

Local Startup

cd ai-infra
cp .env.example .env
# Edit .env, at least set GATEWAY_TOKEN, AI_PROVIDER, and upstream configuration
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
uvicorn gateway.app:app --host 0.0.0.0 --port 8000

When there is no cloud Key, set AI_PROVIDER=mock:

curl http://127.0.0.1:8000/healthz
curl -X POST http://127.0.0.1:8000/v1/chat/completions \
  -H 'Authorization: Bearer test' \
  -H 'Content-Type: application/json' \
  -d '{"model":"mindpaw","messages":[{"role":"user","content":"你好"}]}'

Alternatively, you can use:

docker compose up --build -d

Connecting to a Real Device

The gateway host needs to communicate with the MindPaw's STA network; MindPaw hotspot is only used for the control page.

Open http://192.168.4.1/aiconfig.html:

• API Key: Enter the GATEWAY_TOKEN of the gateway.
• 推理端点 ID: Enter the mindpaw or the model name used in the experiment.
• AI Gateway 地址: Enter http://<网关IP>:8000/v1/chat/completions.

Leaving the Gateway address blank will restore the ESP8266 direct connection to Doubao mode.

Code Entry Points

Contribution Directions

• Implement true asynchronous queues and SLO schedulers.
• Add local small models or Ollama routing.
• Add SQLite/PostgreSQL session storage.
• Increase Prometheus metrics and experiment visualization.
• Establish action safety benchmarks for robot Agents.
• Collect reproducible experimental data under different network and power conditions.

When submitting research-oriented changes, please also include: problem definition, baseline, metrics, ablation experiments, hardware environment, and reproduction experiment commands.