Job Description

Summary:

The Reality Labs Research (RLR) Team brings together a world-class team of researchers, developers, and engineers to create the future of virtual and augmented reality, which together will become as universal and essential as smartphones and personal computers are today. We are developing all the technologies needed to enable breakthrough AR glasses, including optics and displays, computer vision, audio, graphics, brain-computer interfaces, haptic interaction, eye/hand/face/body tracking, perception science, and true telepresence.

The RLR Audio team believes that by redefining human hearing and developing auditory machine intelligence, our research will overcome the barriers of distance, noise, and hearing ability to enhance human connection and communication. Through our Superhuman Communication and Connection initiative, we develop and mature audio technologies for AR glasses and wearables, from early research through tech transfer to the product teams that ship them.

We are seeking a ML Software Engineer to bring state-of-the-art machine learning out of the lab and onto the device. Research models only matter to our prototypes if they run in real time, within tight compute, memory, and power budgets, on the embedded hardware that powers wearables. This role sits at that boundary: you will train and adapt models alongside research scientists, then own the work of optimizing, porting, and running them efficiently on embedded targets.

The ideal candidate is equally comfortable in a PyTorch training loop and in a C/C++ inference runtime on a resource-constrained device. They understand that on-device ML is a co-design problem, where model architecture, quantization, and the target hardware all shape one another, and they can make the trade-offs required to get a research model running on real hardware quickly. Familiarity with audio ML, computer vision ML, or audio DSP is a strong plus, as many of our models operate on multimodal (audio and visual) sensor data.

Our team cultivates an honest and considerate environment where self-motivated individuals thrive. We encourage a strong sense of ownership and embrace the ambiguity that comes with working on the frontiers of research.

We have plenty of researchers who know how to train models, but limited expertise in optimizing and running them on embedded devices. The immediate need is optimizing speech extraction models for on-device inference, but RLR Audio has an ongoing need for this expertise beyond that first project.

A Typical Day involves working closely with research scientists to understand a new model and its deployment constraints, then owning the path to getting it running efficiently on target hardware: training or adapting the model, optimizing, and porting it, profiling on-device, and iterating. You share knowledge and expertise with teammates, contribute directly to technical solutions, and actively participate in team tech meetings.

Minimum Qualifications

Industry experience optimizing and running ML inference on embedded or resource-constrained devices
Experience training ML models using deep learning frameworks such as PyTorch or TensorFlow
2+ years of industry experience writing C/C++
Experience profiling and debugging performance issues across the ML model and the target hardware
Experience working in cross-functional teams

Top 3 must-have HARD skills:

Optimizing and running ML inference on embedded / resource-constrained devices
Training ML models (PyTorch or similar)
C/C++ (2+ years industry experience)

Preferred Qualifications

Experience with ML for audio (e.g., speech enhancement, source separation, keyword spotting, acoustic event detection)
Experience with ML for computer vision (e.g., detection, tracking, segmentation, pose estimation)
Experience with audio DSP fundamentals (e.g., filtering, STFT/filterbanks, beamforming, real-time audio processing)
Experience with on-device inference frameworks and toolchains (e.g. ExecuTorch, TFLite/LiteRT, ONNX Runtime, vendor NPU/DSP SDKs)
Experience with model compression techniques such as quantization-aware training, post-training quantization, pruning, or distillation
Experience writing hardware-specific optimizations (e.g., SIMD/NEON, DSP intrinsics, fixed-point arithmetic)
Experience working in the AR/VR or wearable device space
Ability to gather requirements and work closely with researchers to develop novel solutions
History of writing code to support the execution of research initiatives

Good to have skills:

Audio ML
Computer vision ML
Audio DSP

Responsibilities

Work side by side with research scientists to take ML models from research prototypes to efficient, real-time inference on embedded devices
Train, fine-tune, and adapt ML models with deployment constraints (latency, memory, power, compute) in mind
Optimize models for on-device inference using techniques such as quantization, pruning, distillation, operator fusion, and architecture adjustments
Port, integrate, and profile ML models on embedded targets (e.g. DSPs, NPUs, microcontrollers, mobile SoCs), identifying and resolving performance bottlenecks
Write clean, maintainable, high-performance C/C++ code for on-device inference pipelines and integrate it into real-time prototype systems
Build tools and infrastructure that make it faster for researchers to evaluate model performance and accuracy on target hardware
Coordinate with internal hardware, runtime, and toolchain teams across RL to evaluate and integrate with existing systems
Work collaboratively as part of a multifunctional team where communication, documentation, and teamwork are highly valued
Learn constantly, dive into new areas with unfamiliar technologies, and embrace the ambiguity of AR/VR problem solving

How will performance be measured: While work is collaborative in nature, we’re looking for motivated individuals who can take a goal (“get this model running in real time on this device”), form a plan, make progress independently, raise concerns proactively, and provide input into overall team goals.

SGS Consulting

Latest Jobs

Computer Vision Engineer