Data Engineer

Description

Our client is building a new data platform to support their Modern Sales initiative. This data platform will help sales reps at Constellation gain better access to customer, market, pricing, and real time data. This is a new organization that is being built out. They are hiring a data science team, data enablement, full stack development, and testing teams.

Primary Duties and Accountabilities

• Produce high-quality, secure, and maintainable code/configurations

• Learn and understand business processes with

limited guidance • Work collaboratively as a member of the development team to build best-in-class software solutions in

an agile environment • Support and research issues across all application layers and database

• Identify areas to improve

and scale our Azure architecture and application design

• Ensure code can be deployed using Azure DevOps

• Design and query database tables, views, functions, stored

procedures and batch processes • Develop, implement, and support interfaces that connect our websites, back-end

systems, and various 3rd party cloud solutions, vendors and customers.

• Engage with business partners to gather information, analyze requirements and deliver practical, efficient and

cost-effective solutions that satisfy business needs. Write product specifications and design documentation for assigned

system components.

• Design, develop and/or review complex code or configurations to ensure AI solutions meet the

requirements of functional and technical specifications. Develop and/or review project technical architecture design and

development. Solutions may include but are not limited to the development of various types of applications, web, mobile

app, full stack or integrations hosted on premises data centers or in the cloud. Develop automated Unit tests and/or

automated UI tests.

• Design and build large-scale AI data platforms/products & frameworks for processing high volumes of data, in

real-time as well as batch, that will be used by teams across the organization • Develop complex data algorithms for

AI/ML, data analytics, machine learning, or scientific computing • Utilize a deep understanding of AI/ML/data applications

to translate business needs into technical solutions • Lead team members and provide oversight for less experienced

engineers; Stay on top of the latest technologies and trends. Support development of version control principles (e.g. Git

and working within an agile environment)

• Enhance knowledge of and compliance with preferred technologies, methodologies, standards and policies. Maintain

technical knowledge and business acumen within own discipline or function. Strong debugging and problem-solving skills;

lead peer code reviews. • Provide IT teams and business personnel with technology solutions by weighing advantages of

technology trends, market availability of products, risks and benefits of technology to meet business/IT needs. Participate

in IT architecture review and standards setting. Assist in the development of white papers, conducting presentations as

needed to explain why a technology is being recommended by IT.

Big Data and Integration:

Expertise in handling big data systems and integrating data from diverse sources (e.g., APIs, flat files, databases).

Network and External Connections:

Experience in configuring secure external connections to data sources (e.g., Azure Data Lake Storage, On-prem SQL Server) using service principals, SAS tokens, OAuth, or mounting techniques.

Security and Governance:

Strong understanding of data security, encryption, and compliance standards

Experience implementing data governance frameworks to manage data quality.

Azure Data Factory (ADF):

Proven expertise in using ADF for orchestrating and automating data workflows across cloud and on-premises systems.

• Bachelor’s degree (ex: Computer Science or related discipline) and 5-8 years of relevant experience or minimum 9-12

years of relevant combination of education and work experience.

• Experience with private and public cloud architectures,

pros/cons, and migration considerations

• Minimum of 5 years of RDBMS experience • Experience with JSON, JSON-LD,

XML data structures

· • Experience implementing data pipelines using latest technologies and techniques

• Experience with SDLC products

(Azure DevOps, JIRA, Confluence, GitHub) or similar agile project management tools

• 3-5+ years of hands-on experience in programming languages such as Java 8, c#, node.js, python, SQL, Unix shell/Perl scripting • 5+ years of industry

experience data engineer, with involvement in the data component of the AI/ML lifecycle, and a solid understanding of

applied machine learning topics

• At least 3-5 years of consulting or client service delivery experience on Azure •

Experience handling Structured and unstructured datasets

· • Strong t-SQL skills with