Data Engineer
Our client, an AI-driven company impacting the government sector, is hiring a Mid-to-Senior Level Data Engineer to join their Washington D.C. team remotely. The successful candidate will be responsible for designing, optimizing and scaling their data infrastructure using modern data stack tools and cloud platforms, with a primary focus on Azure.
Responsibilities
-
As the Data Engineer, you will design and optimize data pipelines, storage and infrastructure to ensure scalability and performance.
-
Develop and implement data transformation processes to ensure clean, reliable and accurate data for analytics and AI-driven insights.
-
Integrate data from multiple sources (APIs, flat files, government databases) and ensure efficient data ingestion.
-
Collaborate closely with cross-functional teams (e.g. data scientists, software engineers) to ensure seamless integration of data systems.
-
Communicate complex data concepts to both technical and non-technical stakeholders.
-
Collaborate on branching strategies and development processes to nurture efficient team-based development.
-
Troubleshoot data issues, optimize existing processes and provide proactive solutions to ensure continuous data quality and integrity.
Skillset
-
In-depth understanding of schema design, normalization, indexing and query optimization with hands-on experience in PostgreSQL for data modeling and optimization.
-
Proficient in dbt, SQL and Python for data transformation and cleansing, with experience using orchestration tools such as Airflow or Azure Data Factory.
-
Skilled in ingesting data from diverse sources, including APIs, flat files and government databases like SAM.gov.
-
Familiar with ETL/ELT tools like Airbyte, PyAirbyte and metadata governance tools such as OpenMetadata.
-
Hands-on experience with Azure Data Factory, Azure Blob Storage and Azure Databricks.
-
Experienced in managing data lakehouses with Delta Lake and container orchestration using Kubernetes.
-
Proficient in Python for data manipulation and automation.
-
Strong SQL skills, including the ability to write complex queries using CTEs, window functions, and optimizations.
-
Experience with Git, including branching strategies and collaborative development practices.
-
Knowledge of US government procurement regulations and related data sources is a plus.
Interested? Apply Now!!
49819
SHARE JOB