Frisson Labs is hiring Founding ML Engineers to build AI companions for games that can play like human friends. This involves tackling complex challenges in low-latency response, persistent personality, voice timing, and real-time game competence. They are seeking experts in general ML systems for game AI and specialized ML engineers for audio/speech systems.
A starter prompt for Claude Code, what you'll need, and how to reach them.
You are an expert ML Engineer. Your goal is to scope and build a foundational MVP for an AI companion that can understand game state, make decisions, and respond with synthesized speech in near real-time. Focus on the core challenge of low-latency interaction and basic game competence. Here's the plan: 1. **Define MVP Game Environment**: Start with a very simple, text-based adventure game or a simple 2D environment (e.g., using Pygame or a web-based canvas). The game state should be easily representable as structured data (JSON, dicts). 2. **Game State Understanding**: Implement a component that takes the current game state as input and generates a concise, semantic understanding of 'what is happening'. Use a small, fine-tuned LLM (e.g., a local model or a highly constrained API call) for this, focusing on latency. 3. **Decision Making**: Develop a policy network or a rule-based system that takes the understood game state and decides on the 'best' next action. For the MVP, these actions can be simple commands within the game (e.g., 'move left', 'attack', 'use item'). 4. **Speech Synthesis & Understanding**: Integrate a low-latency Text-to-Speech (TTS) API (e.g., ElevenLabs, Google Cloud TTS) for AI responses and a Speech-to-Text (STT) API (e.g., Whisper API, Google Cloud STT) for player input. Implement basic turn-taking and interruption handling. 5. **Integration & Latency Focus**: Connect all components. Crucially, optimize the entire pipeline for minimal latency from player input to AI response, especially for speech. Prioritize speed over complex reasoning for the MVP. 6. **Deliverables**: A functional prototype of the simple game with the AI companion, demonstrating low-latency interaction. Include a brief technical write-up on the challenges and solutions for real-time performance. Use Python for backend logic and ML, and potentially a simple web framework (Flask/FastAPI) or Pygame for the game environment. Prioritize open-source or easily accessible APIs for initial prototyping.
Frisson Labs | Founding ML Engineer - General, Founding ML Engineer - Audio/Speech | San Francisco, CA ONSITE | Full-time | https://www.frisson-labs.com Frisson Labs is building AI players that play like Discord friends. The hard part is making them respond with low latency, persistent personality, voice timing, and actual game competence: understanding what is happening, deciding what to do, and executing in real time. Open roles: - Founding ML Engineer - General: own the full stack for how AI companions play games: game-state understanding, learned policies/controllers, world models, action representation, MLP/policy heads, memory, evals, data pipelines, inference, and product iteration. Strong fit if you have shipped applied ML systems and want to own messy model behavior all the way from research prototype to live gameplay. - Founding ML Engineer - Audio/Speech: explore and ship low-latency speech systems for real-time play: duplex/streaming voice models, speech understanding, turn-taking, interruption handling, latency/quality tradeoffs, prosody/emotion, and conversational evals. Strong fit if you have deep audio/speech ML experience and care about making voice feel socially
Email founders@frisson-labs.com with 'Founding ML Engineer' in the subject line.
“I've been building and shipping applied ML systems for real-time interaction and am deeply interested in AI companions for gaming. I've developed a prototype demonstrating low-latency game-state understanding and speech integration, which aligns with the core challenges you're tackling. My GitHub and portfolio (link) reflect this focus; I'd be keen to discuss how my expertise can contribute to Frisson Labs.”
Open the original ↗