Software Engineer 3 - Linux/Bash/Python/Apache Airflow/SQL/Jupyter Notebook/NumPy/JSON

About this role

Captivation Software seeks a senior-level software engineer to design and optimize data ingress/egress pathways for large-scale analytics workloads using distributed processing technologies. The role focuses on dataflow architecture, data transport mechanisms, and Apache Spark-based systems to support mission-critical government operations.

What you'll do

• Design and implement efficient data ingress/egress pathways for analytics workloads
• Optimize distributed data processing pipelines using Apache Spark
• Create and maintain Apache Airflow workflows for orchestrating complex data tasks
• Develop automation scripts and tools using Bash and Python
• Ensure data quality, validation, and anomaly detection across systems
• Collaborate on dataflow architecture supporting reliable and scalable processing

What they're looking for

• Linux CLI and command-line tools
• Bash scripting for automation
• Python and Java development
• Apache Airflow (DAG design, scheduling, operators, sensors)
• Apache Spark and distributed big data processing
• SQL (MySQL, MariaDB, PostgreSQL)
• Data wrangling with pandas and NumPy
• Git version control