Skip to main content

Two Six Technologies

Data Collection Engineer

Remote, USA (Remote)From $200kmidAdded today

About this role

Two Six Technologies seeks an experienced Data Collection Engineer to design and maintain distributed web scraping infrastructure that extracts data from complex web ecosystems at scale. You'll build resilient Python-based pipelines, manage containerized workloads on Kubernetes, ensure strict data quality, and overcome anti-bot barriers while handling dynamic content extraction.

What you'll do

  • Design and deploy high-performance distributed web scrapers using Python and Scrapy
  • Extract data from dynamic, JavaScript-heavy websites using browser automation tools
  • Deploy and scale scraping workloads on Kubernetes with optimal resource allocation
  • Define JSON schemas and enforce data validation using Pydantic to maintain data integrity
  • Build and optimize data ingestion pipelines with Elasticsearch for searchable storage
  • Architect end-to-end pipeline workflows managing discovery, extraction, validation, and storage

What they're looking for

  • Python (expert-level)
  • Scrapy and distributed scraping architectures
  • Browser automation (Playwright, Selenium, or Puppeteer)
  • Kubernetes orchestration and container management
  • Elasticsearch indexing and optimization
  • JSON, JSON Schema, and Pydantic validation
  • Proxy rotation and anti-bot circumvention techniques
  • Pipeline workflow design and data engineering

Benefits

  • 100% remote position
  • Work on innovative products solving complex global challenges
  • Collaborative environment with focus on trust and teamwork
  • Opportunity to reverse-engineer sophisticated web systems
Apply on the employer's site →

Opens the official application on the employer’s site. No login required.

Two Six Technologies

Two Six Technologies builds innovative products and training solutions for the intelligence and cybersecurity communities, with a focus on web applications, cyber operations, and simulation environments. The company is hiring for UI/UX developers, test engineers, technical operations engineers, systems engineers, and infrastructure specialists to support product development and advanced training program delivery.

View all jobs at Two Six Technologies

Likely interview questions

  • Walk us through a complex web scraping project where you had to overcome anti-bot protections—what techniques did you use?
  • How do you design distributed scraping architectures to handle millions of pages efficiently, and what trade-offs do you consider?
Unlock all 7 questions free — and practice them live →