Databricks Engineer

Company Description

Merkle is a leading data-driven, technology-enabled, global performance marketing agency that specializes in the delivery of unique, personalized customer experiences across platforms and devices. For more than 30 years, Fortune 1000 companies and leading nonprofit organizations have partnered with Merkle to maximize the value of their customer portfolios. The agency’s heritage in data, technology, and analytics forms the foundation for its unmatched skills in understanding consumer insights that drive people-based marketing strategies. Its combined strengths in performance media, customer experience, customer relationship management, loyalty, and enterprise marketing technology drive improved marketing results and competitive advantage. With 9,600+ employees, Merkle is headquartered in Columbia, Maryland, with 50+ additional offices throughout the US, EMEA, and APAC. In 2016, the agency joined the Dentsu Aegis Network. For more information, contact Merkle at 1-877-9-Merkle or visit www.merkleinc.com.

Job Description

Applicants

• Applicants must be currently authorized to work in the U.S. on a full-time basis. No sponsorship or work transfers are available for this position

Location to work

• Must currently live in and have a permanent residence in the United States 
• Must be able to work in PST

Merkle is looking for experienced Databricks professionals to be part of its Cloud practice. You will be working on industry leading public Cloud platforms such as Amazon Web Services, Microsoft Azure, and Google Cloud Platform to design, develop, and implement next generation marketing solutions leveraging cloud native and commercial/open source big data technologies.

You will be responsible for architecting and implementing very large scale data intelligence solutions around Databricks Lakehouse Architecture.  A solid experience and understanding of architecting, designing and operationalization of large scale data and analytics solutions on Databricks is a must. Good to have professional knowledge of Snowflake, Google BigQuery, AWS Redshift or other Cloud DB.

Key Responsibilities  

• Collaborate with cloud architects, tech leads, front end developers, back end developers, QA, business stakeholders, and project managers to design and build services on Azure/GCP/AWS for applications on the Web or on edge devices.
• You should understand and develop data pipelines and use modern ways of automating data using cloud-based and on-premise technologies
• Good Understanding of Event Based Architectures for Real Time Integration and Stream Analytics.
• Develop reference architectures and design pattern library for typical Cloud based solutions implementations
• Advise on Cloud project set up, security and role based access implementation, and network optimizations
• Conduct training and enablement programs for Merkle’s internal delivery teams
• Hands-on work as part of a global team and coach off shore developers
• Participate in technical interactions with client’s executives and senior management
• Participate in technical customer engagements such as Proof of Concepts, project delivery for key strategic opportunities.
• Develop end-to-end Enterprise grade Concept Solutions, facilitate discussions with business & IT stakeholders and highlighting the outcomes that can be delivered through these solutions.

Qualifications

• 3+ years with SQL and database table design – able to write structured and efficient queries on large data sets
• 3+ years' experience on Databricks on any cloud platform (AWS, Azure or GCP)
• 3+ years' experience implementing Near real time processing using Kafka or other queue mechanism [Confluent or Apache].
• 3+ years' design and/or implementation of enterprise applications
• 2+ years’ experience in orchestrating ETL pipelines using tools like Airflow, Azure Data Factory, NiFi etc….
• Experience in “migrating” on premise workloads to one or more industry leading public cloud(s)
• 2+ years' experience in Scala and Python development, PySpark and Python big data libraries such as pandas, tensorflow, scipy, numpy
• Experience in AGILE development, SCRUM and Application Lifecycle Management (ALM)
• Hands-on experience in full life-cycle of software development or methodology using Agile Scrum/Kanban etc; requirements analysis, design, development testing and implementation
• Able to work with agile teams as they perform feature level design, development, testing, and performance analysis
• Develop the required technical modules/end-to-end solutions for multi layere