Railtown AI is hiring a Head of Developer Relations for their platform focused on AI-powered software development, including developer productivity, backend infrastructure for agents, a Python framework, and evaluation workflows. The role involves shaping the developer narrative and driving adoption through content, demos, docs, community, and open-source growth. This is a full-time, well-paid role with equity.
What they want, where you stand, and the exact résumé edits to qualify.
Biggest lever: Gain dedicated experience in developer advocacy, community building, and scaling a technical audience.
A starter prompt for Claude Code, what you'll need, and how to reach them.
You are an expert full-stack developer. Your task is to conceptualize and outline the MVP for a 'solo-operator friendly' version of an AI agent evaluation and observability platform, inspired by the problem space Railtown AI addresses, but sharply niched for a solo founder. Focus on a specific pain point in agent evaluation that is underserved by existing open-source and commercial tools. The goal is to build a web-based tool where users can upload or define an AI agent's input/output pairs, run predefined or custom evaluation metrics (e.g., correctness, latency, cost), and visualize the results over time.
Your MVP should target the following:
1. **Core Functionality**: Allow users to define a 'test suite' for an AI agent. This includes:
* Input data (e.g., user prompts, context).
* Expected output or criteria for success (e.g., regex match, sentiment score threshold, human feedback collection point).
* Integration points for various LLMs (Anthropic Claude, OpenAI, Gemini via AI SDK v6) and potentially custom agent endpoints.
2. **Evaluation Metrics**: Implement basic metrics like:
* Latency (response time).
* Token usage (input/output counts).
* Correctness (against expected output/criteria).
* Cost estimation per run.
3. **Visualization**: A simple dashboard showing evaluation results over time for an agent, highlighting regressions or improvements.
4. **Tech Stack**: Next.js 16 App Router (React 19), Tailwind v4 for UI, AI SDK v6 with Gemini for LLM integrations, Neon Postgres for data storage. Use Prisma ORM. Deploy to Vercel.
**MVP Slice**: Focus purely on the 'Agent Evaluation' aspect. Users should be able to define a single agent, upload a small CSV of test cases, run evaluations, and see a basic tabular/chart summary of latency, tokens, and a 'pass/fail' based on simple output matching criteria. Assume the agent endpoint is a user-provided URL.
**Build Gate**: A user can define an agent, upload 10 test cases, run evaluations, and see a summary table within 5 minutes. The UI should be responsive and visually clear.The audience for Railtown AI (developers building and evaluating AI agents) is a direct fit for the operator's agent-eval-lab, mcp-kit, ai-usage-monitor, and forge-kit. The operator could showcase how their tools enhance or complement aspects of the Railtown AI stack or address niche gaps.
Railtown AI | Head of Developer Relations | REMOTE (North America preferred) / Vancouver, BC | Full-time | $130k-$180k + equity | https://employmenthero.com/en-ca/jobs/position/railtown-ai-h... Railtown AI is building a unified platform for modern AI-powered software development. Our products span developer productivity engineering, backend infrastructure for agent and app development, a Python framework for production-ready agents, and structured evaluation workflows for responsible AI deployment. We're looking for a Head of Developer Relations to help make Railtown AI the default stack for building, observing, and evaluating AI agents. You'll own the developer narrative across Conductr, Railengine, Railtracks, and Agent Evaluations, then turn that story into adoption through technical content, demos, docs, community programs, open source growth, and direct feedback loops with product and engineering. Good fit if you've spent 5+ years in developer advocacy, DevRel, technical product marketing, or a similar developer-facing role; have scaled a developer platform or technical audience; understand AI tooling, agents, and observability; and can write code, demo live, and communicate
Spend 3-6 months actively contributing to open-source AI agent projects, writing technical blog posts or tutorials, and creating public demos/videos showcasing practical use cases for AI agents and observability tools. Focus on building a small project that exposes observability metrics for an AI agent workflow and writing about its design and implementation.
Standard Next.js deployment
Standard relational database for Next.js
Familiarity with AI SDK v6 for LLM integrations
Familiarity with AI SDK v6 for LLM integrations
Standard AI SDK v6 integration
Standard for Next.js with Postgres
This is a hiring post, not a product idea to clone. The operator should not contact the poster to 'build' this role, but rather assess the market it defines.
“No clear outreach angle for building; this is a job posting. However, the market signals are strong for building a product in this space. If building a related tool, one could say: 'I've developed a specialized AI agent evaluation tool that addresses X pain point your platform might encounter, focusing on [niche feature].'”
Open the original ↗