Lead Software Engineer - AI Data Systems

Location: San Diego
Posted on: June 23, 2025

Job Description:

Upwork ($UPWK) is the world’s largest work marketplace, connecting businesses with highly skilled professionals worldwide. From entrepreneurs to Fortune 100 enterprises, companies trust Upwork’s platform to access expert talent, leverage AI-powered work solutions, and drive meaningful business outcomes. Upwork’s AI-powered platform has facilitated over $20 billion in economic opportunity for professionals worldwide. With professionals spanning 10,000 skills, including AI and machine learning, software development, sales and marketing, customer support, finance and accounting, and more, Upwork empowers businesses of all sizes to scale, innovate, and build agile teams. We’re looking for a Lead Software Engineer (AI Data Systems) to join our growing AI team and help build the critical infrastructure that powers the future of intelligent, agent-driven systems at Upwork. You’ll be responsible for collecting high-quality training data, building scalable featurization pipelines, and delivering performant systems to support model training and inference at scale. This role is built for Explorers, engineers who thrive in ambiguity, move fast with instinct, and are energized by greenfield challenges. Instead of narrowly scoped specialists, we’re hiring T-shaped engineers: individuals with a breadth of skills across the stack and deep expertise in at least one technical area. You’ll be one of our foundational hires, pushing past the known, building fast in Python, and helping us shape our technology and our team. Experience in startups, AI/ML research environments, or similarly dynamic settings is essential, as we’re solving problems that demand speed, curiosity, and adaptability. If you’re passionate about AI infrastructure, thrive in early-stage environments, and want to help us set the standard for how agentic systems are built and deployed at scale, this is your opportunity. Responsibilities • Design and implement systems to collect and curate high-quality training datasets for supervised, unsupervised, and reinforcement learning use cases. • Build scalable featurization and preprocessing pipelines to transform raw data into structured inputs for AI/ML model development. • Partner with ML engineers and researchers to define data requirements and production workflows that support LLM-based agents and autonomous AI systems. • Lead the development of infrastructure that enables experimentation, evaluation, and deployment of machine learning models in production environments. • Support orchestration and real-time inference pipelines using Python and modern cloud-native tools, ensuring low-latency and high availability. • Mentor engineers and foster a high-performance, collaborative engineering culture grounded in technical excellence and curiosity. • Drive cross-functional alignment with product, infrastructure, and research stakeholders, ensuring clarity on progress, goals, and architecture. What it takes to catch our eye • Strong software engineering background with deep experience in building data collection, transformation, and featurization pipelines at scale. • Proficiency in Python, including async programming and concurrency tools, as well as data-centric frameworks such as Pandas, Spark, or Apache Beam. • Familiarity with ML model development workflows and infrastructure, including dataset versioning, experiment tracking, and model evaluation. • Experience deploying and scaling AI systems in cloud environments such as AWS, GCP, or Azure. • Proven success operating in highly ambiguous environments such as research labs, startups, or fast-paced product teams. • A track record of working with or alongside high-caliber peers in top engineering teams, research groups, or startup ecosystems. • Growth mindset, strong communication skills, and a commitment to inclusive collaboration and continuous learning. At Upwork, you’ll shape the future of work for a global, remote-first workforce, creating economic opportunities for professionals worldwide. While we have a physical office in Palo Alto, we currently hire full-time employees in 21 U.S. states, making it easier than ever to join our mission from wherever you call home. Our culture is built on trust, risk-taking, customer focus, and excellence, all in service of our core mission: to create economic opportunities so people have better lives. We embrace authenticity and inclusion, encouraging everyone to bring their whole selves to work. Personal and professional growth is a priority here, supported through development programs, mentorship, and our Upwork Belonging Communities.

Keywords: , Malibu , Lead Software Engineer - AI Data Systems, IT / Software / Systems , San Diego, California