Lead Big Data Engineer

Job Description:

POSITION:                 Lead Big Data Engineer

JOB LOCATION:       208 S. Akard St., Dallas, TX 75202[and various unanticipated locations throughout the U.S.; may work from home]

DUTIES: Lead Big Data Engineer needed by AT&T Services, Inc.  in Dallas, TX [and various unanticipated locations throughout the U.S.; may work from home] to provide Application Security Log Alarming (ASLA) and Advanced Secure Telecommunications Routing Architecture (ASTRA) support in the cloud along with Upstart. Work with Halo effort, driving solutions to aid in migration from Upstart to help drive role-based access control and least privilege access control to the data and the platforms in conjunction with the public cloud and governance teams. Design, deploy and manage the various Platform as a Service (PAAS) and Software as a Service  (SAAS) platforms available and manage in the cloud by CDO including Databricks and Snowflake. Define data requirements, gather and mine large volumes of structured and unstructured data at scale, and validate data by running various data tools in the Big Data Environment. Support the standardization, customization and ad-hoc data analysis and develop the mechanisms to ingest, analyze, validate, normalize and clean data. Implement statistical data quality procedures on new data sources and by applying rigorous iterative data analytics, supports Data Scientists in data sourcing and preparation to visualize data and synthesize insights of commercial value. Work with Big Data Policy and security teams and legal teams to create data policy and develop interfaces and retention models which requires synthesizing or anonymizing data. Develop and maintain data engineering best practices and contribute to insights on data analytics and visualization concepts, methods and techniques. Work to drive down cloud costs for individual job runs with the potential use of GPU’s in place of CPU based workloads. Work with peers and vendors to help determine potential deployment and use of ICEBerg storage across various platforms including Databricks, Snowflake and Palantir. Utilize Azure cloud experience designing, developing, testing and with the administrative production of cloud-based environments. Work to define network solutions and structures that define and map out the overarching environment on cloud-based solutions. Work with Azure Keyvault, secret scopes, private endpoints, Azure AD, Active Directory (AD), and Active Directory Domain Services (AADDS). Work with HDI,  Azure’s cloud-based implementation of Hadoop, ADLSg2, Delta Live Tables, and pipelines. Manage and configure requirements of Databricks on Azure to ensure platform integrity. Design, develop, and configure of Realtime UI monitoring utilizing Prometheus and Grafana UI with Databricks to understand and drive down potential costs.

REQUIREMENTS: Requires a Bachelor’s degree, or foreign equivalent degree, in Electronic Engineering, Engineering or Computer Science, and 2 years of experience in the job offered, or 2 years of experience in a related occupation utilizing Azure cloud experience designing, developing, testing and with the administrative production of cloud-based environments; working to define network solutions and structures that define and map out the overarching environment on cloud-based solutions; working with Azure Keyvault, secret scopes, private endpoints, Azure AD, Active Directory (AD), and Active Directory Domain Services (AADDS); working with HDI,  Azure’s cloud-based implementation of Hadoop, ADLSg2, Delta Live Tables, and pipelines; managing and configuring requirements of Databricks on Azure to ensure platform integrity; designing, developing, and configurating of realtime UI monitoring utilizing Prometheus and Grafana UI with Databricks to understand and drive down potential costs.

Our Lead Big Data Engineers earn between $156,550 - $231,100 yearly. Not to mention all the other amazing rewards that working at AT&T offers.

Joining our team comes with amazing perks and benefits:

· Medical/Dental/Vision coverage

· 401(k) plan

· Tuition reimbursement program

· Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays)

· Paid Parental Leave

· Paid Caregiver Leave

· Additional sick leave beyond what state and local law require may be available but is unprotected

· Adoption Reimbursement

· Disability Benefits (short term and long term)

· Life and Accidental Death Insurance

· Supplemental benefit programs: critical illness/accident hospital indemnity/group legal

· Employee Assistance Programs (EAP)

· Extensive employee wellness programs

· Employee discounts up to 50% off on eligible AT&T mobility plans and accessories