Software Engineering Expert
- Salary
- $60 - $90 per hour
- Level
- Senior
- Work type
- Remote · US only
Not stated in the posting: visa.
Open. First seen 3 October 2026.
About the role
This role is for one of our clients
Compensation: $60-$90 per hour
Join a pioneering AI initiative focused on building the next generation of evaluation benchmarks for frontier AI models. We are seeking experienced Software Engineers to design and validate sophisticated engineering tasks that mirror real-world software development challenges.
In this role, you will create complex, multi-step software engineering benchmarks that require implementation, debugging, environment configuration, and technical reasoning. Working alongside AI researchers, you'll help identify where advanced AI coding systems succeed, where they fail, and how benchmark quality can be continuously improved.
This is a fully remote, full-time engagement requiring approximately 35 hours per week.
Requirements
Key Responsibilities
- Design realistic, multi-step software engineering challenges based on practical development workflows, including implementation, debugging, system configuration, testing, and documentation.
- Develop comprehensive reference solutions in Python, ensuring each task is reproducible, verifiable, and supported by appropriate validation checks.
- Leverage AI coding assistants as part of your development workflow while evaluating their strengths, limitations, and failure modes on complex engineering tasks.
- Review benchmark tasks created by fellow experts, providing detailed feedback on technical correctness, clarity, completeness, and level of difficulty.
- Analyze AI-generated solutions to identify implementation errors, debugging challenges, reasoning gaps, and opportunities for improving benchmark quality.
- Collaborate closely with AI researchers and engineering teams to refine evaluation methodologies and maintain consistently high technical standards.
Required Qualifications
- Master's degree, PhD, or equivalent practical experience in Computer Science, Software Engineering, or another STEM discipline involving substantial programming and technical problem solving.
- Minimum 1 year of professional experience in software engineering, research engineering, applied research, or another technically intensive engineering role.
- Strong hands-on proficiency in Python, including software design, scripting, debugging, testing, and writing clean, maintainable code.
- Solid experience using Git, integrated development environments (IDEs), and modern software development workflows.
- Strong understanding of software architecture, debugging methodologies, and engineering best practices.
- Experience using AI coding assistants, prompt engineering techniques, or AI agent workflows is preferred.
- Experience with AI evaluation, benchmark development, AI training, or task authoring is highly desirable.
- Excellent analytical thinking, creativity, attention to detail, and the ability to solve complex, open-ended technical problems independently.
- Strong written communication skills for documenting technical solutions and engineering decisions.
- Ability to commit approximately 35 hours per week on a consistent basis.
Preferred Qualifications
- Experience developing complex software systems across backend, infrastructure, automation, or developer tooling.
- Familiarity with large language models, AI-assisted software development, or autonomous coding agents.
- Background in benchmark design, software quality engineering, or technical evaluation frameworks.
- Contributions to open-source projects, technical publications, or advanced engineering initiatives.
Why Join
- Help shape how next-generation AI coding systems are evaluated against real-world engineering challenges.
- Collaborate with leading AI researchers developing frontier evaluation benchmarks.
- Apply your software engineering expertise to improve AI reliability, technical reasoning, and coding performance.
- Contribute directly to benchmark development that influences the evolution of advanced AI systems.
- Enjoy the flexibility of a fully remote engagement while working on impactful AI research initiatives.
Equal Opportunity
We are committed to fostering an inclusive and diverse environment where all qualified applicants receive equal consideration. Reasonable accommodations are available throughout the application and engagement process.
Contract & Engagement Details
- Independent contractor engagement.
- Fully remote with flexible working hours.
- Expected commitment of approximately 35 hours per week.
- Project duration may be extended, shortened, or concluded based on project requirements and individual performance.
- Work does not require access to confidential or proprietary information from any current or former employer.
- Payments are issued weekly based on approved work completed.
- At this time, we are unable to support H1-B or STEM OPT candidates.
The team: AI Training HIRES STEADILY
This role is one of 172 open in AI Training. 51 were posted in the last 30 days and the oldest in April: a team that hires all year rather than one being built.
WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS
Python (52 of 172), Machine Learning (41 of 172), Generative AI (31 of 172), LLM (31 of 172), Git (18 of 172), C++ (15 of 172), Excel (15 of 172)
OTHER ROLES ON THIS TEAM
- Hardware Engineering Expert - AI Training & Evaluation posted Nov 18
- DevOps / SRE Engineer (Kubernetes, Terraform), AI Expert posted Nov 18
- Film Colorist / Finishing Editor (Resolve, Baselight), AI Expert posted Nov 18
- Senior Character Animator (Film, VFX, Games), AI Expert posted Nov 18
- Software Engineer (Python, Java, C++, Go, Rust), AI Expert posted Nov 18
- VFX Supervisor / Lead Compositor (Nuke, Flame), AI Expert posted Nov 18
- Aerodynamics Expert - AI Training & Evaluation posted Nov 18
- Competitive Data Scientist (Kaggle) - AI Training & Evaluation posted Nov 18
- Mechanical Engineering Expert - AI Training & Evaluation posted Nov 18
- 3D Artist - Modeling, Rigging, Animation & AI Training posted Nov 17
- AI Safety, Responsible AI & Cybersecurity Specialist (AI Evaluation) posted Nov 17
- Brand & Advertising Art Director - Brand Identity, Campaigns & AI Evaluation posted Nov 17
- Commercial Music Production & Audio Post-Production Expert (AI Training) posted Nov 17
- Machine Learning & Scientific Computing Engineer (AI Evaluation) posted Nov 17
- Presentation, Editorial & Document Design Specialist (AI Evaluation) posted Nov 17
- Senior Luxury Automotive Strategy Leader, AI Expert posted Nov 17
- Senior SWE - Python / TypeScript posted Nov 17
- Software Engineer - Core Contributor posted Nov 17
- AI Engineer posted Nov 17
- Architectural Design & 3D Visualization Expert (AI Training) posted Nov 17
- Board-Certified Anaesthesiologist, AI Expert posted Nov 17
- Board-Certified Cardiologist, AI Expert posted Nov 17
- Board-Certified Dermatologist, AI Expert posted Nov 17
- Board-Certified Radiologist, AI Expert posted Nov 17
- Formal Mathematics & Theorem Proving Specialist (AI Evaluation) posted Nov 17
- Economists - Economic Analysis for AI Training posted Nov 16
- P&C Insurance Professionals: Paid AI Training Project posted Nov 16
- Legal Research Attorney - AI Training posted Nov 16
- Customer Support & Success Domain Expert - AI Model Training posted Nov 14
- Subject Matter Experts: ML Model Evaluation (PhD) posted Nov 13
- Senior Motion Designer (After Effects), AI Expert (Remote, US) posted Nov 12
- Commercial Video Editor (Premiere, Resolve), AI Expert (Remote, US) posted Nov 12
- ML Infrastructure and Performance Engineer, AI Expert (Remote) posted Nov 12
- Board-Certified Neurologist, Medical AI Expert (Remote, US) posted Nov 12
- Board-Certified Orthopedic Surgeon, Medical AI Expert (Remote, US) posted Nov 12
- Electrical Engineer, AI Expert (Full-Time, US) posted Nov 12
- Embedded Systems and Firmware Engineer, AI Expert (Remote, US) posted Nov 12
- Licensed Clinician (MD, NP, PA, RN, PharmD), Healthcare AI Expert (Remote, US) posted Nov 12
- Mechanical Engineer, AI Expert (Full-Time, US) posted Nov 12
- PhD Economist, AI Research Expert (Remote) posted Nov 12
- PhD Genomics and Cell Biology Scientist, AI Expert (Remote, UK or US East Coast) posted Nov 12
- Software Engineer - Backend posted Nov 12
- PhD Mathematician or Statistician, AI Expert (Remote, US) posted Nov 12
- PhD Pharmacologist or Toxicologist (Drug Discovery), AI Expert (Remote UK or US) posted Nov 12
- Game Developer (Godot Expert) posted Nov 12
- Finance Professional from a Top-Tier Firm, AI Expert (Remote, US) posted Nov 12
- PhD Physicist, AI Expert (Remote, US) posted Nov 12
- Electronics Engineer (DesignSpark PCB Specialist) posted Nov 12
- PhD Biologist (Life Sciences), AI Expert (Remote, US) posted Nov 12
- PhD Chemist, AI Expert (Remote, US) posted Nov 12
- PhD Synthetic and Analytical Chemist, AI Expert (Remote, UK or US East Coast) posted Nov 12
- Pod lead - Scientific Computing / Research Engineering Expert - Earth Sciences posted Oct 19
- Senior Software Engineer posted Oct 19
- Senior SWE - Python / TypeScript posted Oct 19
- Agentic Life Science Expert posted Oct 19
- Backend Security Engineer posted Oct 19
- Engineering Expert posted Oct 19
- Sr. Full-Stack Software Engineer posted Oct 19
- Software Engineer (Open Source Contributor) posted Oct 19
- Scientific Computing / Research Engineering Expert - Earth Sciences posted Oct 19
- Business Analyst: US Raters posted Oct 19
- Competitive Data Scientist (Kaggle) posted Oct 19
- Junior Data Scientist posted Oct 19
- Junior Software Engineer posted Oct 19
- Business Analyst - US - Educational Raters - Gemini posted Oct 19
- Business Analyst: Non-US Raters posted Oct 19
- Insurance Expert - AI Training & Evaluation posted Oct 8
- Biology / environmental science Evaluator posted Sep 17
- Biotech Investment Analyst - Drug Asset & Company Assessment posted Sep 17
- Data Scientist Talent Network posted Sep 17
- Equity Research Expert - Single-Name Analyst Publications posted Sep 17
- Engineering & Software Domain Expert posted Sep 17
- Management Consultant Talent Network posted Sep 17
- Multilingual Inpatient Hospitalist (MD) - Clinical Documentation & AI Evaluation posted Sep 17
- Finance Domain Expert - AI Training & Evaluation posted Sep 17
- Analyst / Insights Experts posted Sep 17
- MLOps Engineer, LLM Systems (Serving, GPU Kernels, Profiling) posted Sep 17
- Software Engineer, Go - Codebase Q&A posted Sep 16
- Biology / environmental science Evaluator posted Sep 16
- Applied History & Political Science Benchmark Specialist posted Sep 16
- Software Engineer, Python - Codebase Q&A posted Sep 16
- Software Engineer, C - Codebase Q&A posted Sep 16
- AI Safety Experts - English & Finnish posted Sep 16
- Biology PhD Coding Experts posted Sep 14
- STEM PhDs and Technical Domain Experts posted Sep 14
- Physics PhD Coding Experts posted Sep 14
- Material Science PhD Coding Experts posted Sep 14
- Chemistry PhD Coding Experts posted Sep 14
- Data quality / CRM operations Evaluator posted Sep 14
- Mathematics PhD Coding Experts posted Sep 14
- FP&A & Treasury Analyst posted Sep 14
- Electrical Engineering Expert posted Sep 8
- Technical Accounting & SEC Reporting Specialist posted Sep 8
- Incident management / reliability / SRE Evaluator posted Sep 8
- Engineering / manufacturing / technical operations Evaluator posted Sep 8
- Mechanical Engineering Expert posted Sep 8
- Financial Crime, AML & KYC Analyst posted Sep 8
- Generalist (AI Training Specialist) posted Sep 7
- Compliance / regulatory response with financial-services AI Evaluator posted Sep 2
- Cloud / DevOps Engineer (Infra & IaC) posted Sep 2
- Materials Science Domain Expert posted Sep 2
- Product / Platform Experts posted Sep 2
- Engineering / Platform Professionals posted Sep 1
- Product / Artifacts Experts posted Sep 1
- Applied Computer Science Benchmark Specialist posted Aug 17
- Sales Engineering Expert posted Aug 17
- UI / UX Design Expert posted Aug 17
- Professional Design Experts posted Aug 15
- Project Coordinator - AI & Data Projects posted Aug 15
- Senior Software Engineer, Full Stack (Python, Java, Rust, C#, C++) posted Aug 14
- Legal Domain Expert - AI Training & Evaluation posted Aug 14
- Software Engineer, Full Stack (Python, Java, Rust, C#, C++) posted Aug 14
- Senior Design Expert - Paid AI Design Research Study posted Aug 14
- QA/Test Engineer posted Aug 14
- Biology Expert (PhD) - AI Safety posted Jul 30
- Electrical Engineering Expert posted Jul 30
- Mechanical Engineering Expert posted Jul 30
- Chemical Engineering Expert posted Jul 30
- Machine Learning Engineer - Model Evaluation & Experimentation posted Jul 30
- ASIC/SoC Design & Verification Engineer posted Jul 29
- Data Engineer - Finance posted Jul 29
- MLOps Engineer (JAX, PyTorch, Pallas/Triton) posted Jul 29
- Performance Engineer (C++, Python, Rust) posted Jul 29
- Computer Vision Expert posted Jul 29
- Machine Learning & NLP Expert posted Jul 29
- AI Rater Guidelines Writer (Linguist / Instructional Designer) posted Jul 29
- Data Science & Quantitative Analysis Expert posted Jul 29
- QA/Test Engineer posted Jul 29
- Data Science Expert posted Jul 28
- Architect (CAD & Construction Documentation) posted Jul 14
- Frontend Software Engineer posted Jul 10
- AI Evaluation Specialist posted Jul 9
- Technology AI Evaluation Expert posted Jul 9
- Southern American English Voice Actor (AI Speech & Voice Modeling) posted Jul 9
- Legal AI Evaluation Expert posted Jul 9
- Software Engineering Expert (AI Training) posted Jul 9
- Equity Research AI Specialist posted Jul 8
- Internal Medicine AI Evaluation Specialist posted Jul 8
- Medical Expert AI Evaluation Specialist posted Jul 8
- Game Tools & Modding Engineer posted Jul 3
- Biology Subject Matter Expert (AI Training & Scientific Content) posted Jul 3
- Chemistry Subject Matter Expert - AI Training (Remote) posted Jul 3
- AI Safety & Red Teaming Specialist posted Jul 3
- Medical Subject Matter Expert (AI Training & Documentation) posted Jul 3
- Engineering Subject Matter Expert (AI Training) posted Jul 3
- Computer Science Subject Matter Expert (AI Training) posted Jul 3
- Financial Documentation Expert (AI Training) posted Jul 3
- Physics Subject Matter Expert (AI Training) posted Jul 3
- Mathematics Subject Matter Expert (AI Training) posted Jul 3
- Network Engineer - Data for Autonomous Systems annotation posted Jun 24
- Management & Strategy Consultants (MBB/Big 5) AI Trainer posted Jun 24
- CUDA Engineering Expert posted Jun 24
- Cybersecurity SWE - AI Safety posted Jun 24
- Mechanical Engineer posted Jun 5
- US Tax Law AI Specialist (CPA / EA / CA) posted May 14
- AI Tax Compliance & Legal Interpretation Expert posted May 14
- Physics AI Specialist posted May 5
- Clinical AI Specialist posted May 5
- Investment Banking AI Specialist posted May 5
- Tax AI Expert posted Apr 21
- Capital allocation AI Expert posted Apr 21
- Insurance Underwriting AI Expert posted Apr 21
- Financial modeling AI Expert posted Apr 21
- Compliance and controls AI Expert posted Apr 21
- Insurance Policy Administration AI Expert posted Apr 21
- Risk & Actuarial AI Expert posted Apr 21
- Financial reporting AI Expert posted Apr 21
- Strategic finance AI Expert posted Apr 21
- Financial planning & analysis (FP&A) AI Expert posted Apr 21
- Insurance Claims Management AI Expert posted Apr 21
- Ledger and reconciliation AI Expert posted Apr 21
From the team and dates on Weekday AI's own job board.
See Software Engineer salaries in the United States →
Similar jobs
- Software Engineer, Vercel Agent at Vercel
Shares 3 skills: Prompt Engineering, LLM, Python · Senior level · same country - Sr AI Software Engineer at The Trade Desk
Shares 3 skills: Prompt Engineering, LLM, Python · Senior level · same country - Data Scientist at TEGNA Inc.
Shares 4 skills: Prompt Engineering, Git, LLM… · Senior level · same country - Senior Consultant, Gen AI at Lovelytics
Shares 4 skills: Prompt Engineering, Git, LLM… · Senior level · same remote region - Sr. Forward Deployed Data Engineer at Natera
Shares 4 skills: Prompt Engineering, Git, LLM… · Senior level · same remote region - VFX Supervisor / Lead Compositor (Nuke, Flame), AI Expert at Weekday AI
Also at this company
More jobs like this
Is this your posting? Email [email protected] to have it removed.