ETS Data Engineer III

Develop strategies for data engineer to deliver enterprise data products solutions in Cloud. Responsible for designing, building, integrating data from various sources, and build data streaming solutions specializing in near real time data processing. Develop data consumption patterns to share data with internal/external channels, while ensuring they are easily accessible, with the goal of optimizing the performance of Navy Federal’s big data ecosystem. Recognized as an expert with a specialized depth and/or breadth of expertise in streaming data engineering solutions. Solves highly complex problems; takes a broad perspective to identify solutions. Leads functional teams or projects and works independently.  

This position is eligible for the TalentQuest employee referral program. If an employee referred you for this job, please apply using the system-generated link that was sent to you.

 

• Provide Data Engineering solutions in Azure Cloud.
• Work with business leadership to understand data requirements; propose and develop solutions that drive business objective.
• Designing and implementing robust, scalable, and fault-tolerant data streaming architectures.
• Developing and maintaining end-to-end data pipelines for real time data ingestion and processing.
• Collaborating with cross-functional teams to understand business requirements and translating them into efficient streaming solutions.
• Monitoring and optimizing streaming data performance to ensure the low latency and high throughput.
• Design and implement streaming data workflows to process data on cloud data sources, monitor data quality and implement related controls. 
• Ability to prioritize intake efforts and issue resolution with data engineers. 
• Interpret data presented in models, charts, and tables and transform it into a format that is useful to the business and promotes effective decision making.
• Recognize potential issues and risks during the analytics project implementation and suggest mitigation strategies. 
• Document best practices for data models, data loads, query performance and enforce with team members.
• Evaluate existing schema designs on multiple environments, improve data models, and implement optimizations. 
• Apply engineering principles into the design and enhancement of new and existing data systems.
• Ensure the security and integrity of system and product solutions including compliance with Navy Federal, industry engineering and Information Security principles and practices.
• Present clear, organized, and concise information to all audiences through a variety of media to enable effective business decisions.
• Perform other duties as assigned.

• Bachelor’s degree in information systems, Computer Science, Engineering, or related field, or the equivalent combination of education, training, and experience
• Proficiency in programing language such as Python & Spark.
• In-depth knowledge of streaming data concepts and architecture.
• Expertise in working with distributed systems and streaming frameworks with focus on Apache Kafka and Apache Flink.
• Expertise with real time data processing in Cloud (Azure Event Hubs, Azure Stream Analytics, Azure Functions or similar).
• Problem-solving skills to address the issues related to data consistency and scalability.
• Experience in Data Lake (Azure or similar), ETL tools (Azure Data Factory, Azure Databricks or similar), Data Warehouse (Azure Synapse or similar), and Big Data skills (Spark, etc.) for querying and manipulating datasets.
• Experience in Data Modeling, Data Governance (Azure Purview or similar), Data Quality and Data Integration.
• Experience in other programming languages (SQL, Java, etc.) to process data.
•  Experience in integrating structured, semi-structured and unstructured data sources to deliver analytics solutions using Data Lake, Data Warehouse, and NoSQL  Data Hubs (Azure Cosmos DB or similar).
• Experience with processing various file types in Data Lake (Delta, Parquet or similar).
• Experience in monitoring performance of data workflows and infrastructure through use of monitoring tools (Azure Monitor or similar) and enabling alerts for prompt resolution.
• Experience with DevOps toolset (Azure DevOps or similar) to collaborate and deploy code.
• Enable new technology development by conceptualizing solutions and estimating based on minimal requirements.
• Demonstrates change management and excellent communication skills.
• Understands the concepts and application of data mapping and building requirements.
• Ability to communicate end to end data outcomes visually.
• Skilled in managing the process between updating and maintaining data source systems and implementing data related r