Embedded AI Engineer, On-Device Models

About this role

Deepgram seeks an Embedded AI Engineer to optimize and deploy speech AI models on resource-constrained devices like phones, wearables, and IoT hardware. You'll handle model optimization, runtime development, and integration with edge inference platforms to enable fast, accurate, offline voice experiences on embedded systems.

What you'll do

• Optimize Deepgram's speech models through quantization, pruning, distillation, and compilation for embedded targets
• Write performance-critical C, C++, and Rust code for embedded and real-time operating systems (FreeRTOS, Zephyr)
• Integrate with edge inference runtimes and vendor NPU/DSP toolchains for efficient hardware mapping
• Design model packaging, deployment pipelines, and over-the-air update mechanisms for connected devices
• Build benchmarking and validation frameworks to measure latency, accuracy, power, and memory across target hardware
• Partner with silicon vendors on SDK integration and performance tuning for new chipsets

What they're looking for

• Embedded systems and low-power device optimization
• Model optimization techniques (quantization, pruning, distillation, operator fusion)
• C, C++, and/or Rust programming
• Edge ML inference runtimes and compiler toolchains
• Real-time operating systems (FreeRTOS, Zephyr, or similar)
• Performance profiling and benchmarking
• NPU/DSP integration and hardware accelerators
• Familiarity with speech/audio processing or deep learning architectures

Benefits

• Remote work in the USA
• Work on cutting-edge Voice AI at the frontier of on-device inference
• Flexible seniority level (individual contributor to staff engineer)
• AI-first culture with tools and support for continuous innovation
• Opportunity to influence model architecture decisions alongside Research teams

Likely interview questions

• Tell us about a time you optimized a complex model to run on a resource-constrained device—what trade-offs did you make between accuracy, latency, and power?
• How have you approached model quantization or compression in past projects, and which techniques have been most effective for your use cases?