About

This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more.

Role Description

We are seeking a passionate and versatile Senior Data Engineer to join our Ag data team. In this role, you will design solutions for large-scale complexity problems in data architecture and engineering while mentoring team members and driving continuous improvement. You'll work on multi-disciplined teams in a collaborative environment with opportunities to play key roles in product planning, design, prototyping, and execution. Role is remote, but candidate must live near Austin or Houston area.

• Enable our DTN partners and customers through the delivery of trusted data sets and services from complex data sources and environments, supporting their data infrastructure needs, and solving technical issues with the data or environments.
• Be a key contributor in the delivery of our modern data and analytic architecture through execution and delivery of data management, data integration, and data acquisition patterns and pipelines.
• Design solutions for large-scale complexity data problems that are simple and easy to understand by others.
• Drive continuous improvement of internal processes, patterns and best practices to improve data quality, data management, security, performance, and team success.
• Ensure DTN's data is governed and secured as part of solution delivery.
• Contribute to reference documentation to ensure reuse and consistency in how data is acquired, transformed, persisted, and consumed analytically at DTN through enterprise standard patterns, SODs, and reference architectures.
• Directly and indirectly mentor other data engineers to ensure team and department advancement.
• Collaborate with external teams to identify, triage, and resolve data quality issues.

Qualifications

• 5+ years of experience in data modeling and data engineering (pipelines, ingest, ETL/ELT), SQL, and Python for API and data development
• 5+ years of cloud experience (multiple vendors preferred) and cloud data platforms (Snowflake, AWS Native)
• Strong experience with databases including Oracle, Postgres, and MySQL
• Experience in automating data pipelines
• A customer centric mindset where you understand the benefits to the customer of timely high quality data
• Excellent communication skills across diverse audiences, leadership skills, and the ability to facilitate technical conflict resolution

Requirements

• Experience with modern tools for data transformations and automation such as PySpark, Jupyter, Hadoop, Spark, Apache Airflow, Argo Workflows, and AWS Step Functions
• Experience with AWS tooling like Athena, Lambda functions, Glue, RDS Postgres, Redis
• A quality and test-driven mindset tied with experience with data validation and test automation frameworks (e.g., pytest)
• Strong knowledge of best practices for data privacy, security, lifecycle management, and governance
• Experience in devops (CI/CD tools and pipelines) and Agile/SCRUM methodologies
• Proficiency in Linux/Unix command-line operations
• Hands-on experience with GIS/Spatial data

Benefits

• The targeted hiring base pay range for this position is between $101,250 and $151,000
• This role may also be eligible for market competitive variable pay and benefits.