Principal Data Engineer

Company: Comcast Corporation
Location: Philadelphia
Posted on: November 14, 2024

Job Description:

Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. Join us. You'll do the best work of your career right here at Comcast. (In most cases, Comcast prefers to have employees on-site collaborating unless the team has been designated as virtual due to the nature of their work. If a position is listed with both office locations and virtual offerings, Comcast may be willing to consider candidates who live greater than 100 miles from the office for the remote option.)Job SummaryThe Technology, Product, Experience (TPX) organization works at the intersection of media and technology and our innovative teams are continually developing and delivering products and next-generation technologies that transform the customer experience. The Comcast Network and CONNECTivity (CONNECT) organization is a highly agile, fast-paced, dedicated group at the forefront of change focused on innovating, building, and operating the best in class, most reliable access network for our customers.

We are looking for a Senior Data Engineer (Engineer 5) with Neo4j experience to join our Philadelphia-based team. As the senior member of the engineering team, you also influence coding practices, architecture, and team maturation.Job DescriptionTechnology Environment:ROCI [Routing of Critical/Cable Infrastructure] is an enterprise graph data platform that acts as a logical and physical topology graph for Comcast's entire access network from the headend (CMTS) to customer (CPE/Cable Modems). To build this topology graph, ROCI consumes data from a number of BSS/OSS, construction, provisioning, planning, and monitoring platforms to create linkages previously unseen across the company. Our unique enrichment methodology provides insight into our plant, utilization, and routing that will enable increased velocity in network expansion and provide new opportunities to pre-emptively prevent customer affecting events. Our ultimate vision is to build a digital twin that seamlessly represents our physical outside plant as observed by the eye and the logical representation of the network traffic flowing through those components.This team is based in our cutting-edge in downtown Philadelphia. We are in office on a hybrid schedule and this role is not eligible for remote/virtual employment.Core Responsibilities:

• Design, develop, test, and maintain big data ETL's
• Presents and defends architectural, design, and technical choices to internal audiences.
• Assists leadership/architects throughout the design process and with technical guidance regarding practices, procedures, with focus on scale and long term sustainability
• Responsible for creating and maintaining large Graph Data platforms.
• Well versed with ingestion and processing large data sets and provide actionable recommendations using data analytics.
• Develop software systems in an Agile development environment and on AWS cloud-based platform using Python, Spark, Scala, Databricks, NEO4J, and AWS
• Contributes to the development of application metrics & work towards improving them.
• Develop automated processes that identify patterns and make decisions with minimal human intervention.
• Continually works on improving the quality of codebase. -
• Advocating for improvements to data quality, security, and performance of the platform.--------- -
• Work with and mentor junior and Senior Software Development Engineers. -
• Collaborate with data scientists and architects to build cross functional solutions to current and future business imperatives
• Displays in-depth knowledge of and ability to apply software design, and redesign skills.
• Integrates knowledge of business and functional priorities. -Required Skills:
  • Proficient in Python, with good knowledge of concurrent processing
  • Hands on experience in developing complex ETL applications with spark and SQL
  • Good to have knowledge of data-engineering systems such as data lake, file-formats, batch & stream processing (Kafka). Databricks experience is a must.
  • Experience in application development using Docker, Kubernetes, especially at high-scale is a bonus
  • Experience with CI/CD tools such as Git, Jenkins, Concourse
  • Experience with Graph DBs - Neptune/Neo4j
  • Knowledge of AWS ecosystem - S3, Lambda, EC2, EKS, Cloudwatch, Kinesis, SNS, SQS
  • Strong communication and debugging skills required
  • Experience with observability tools such as ELK, Prometheus, Grafana, Opentelemetry
  • Highly proficient in implementing and building practices around unit/integrating testing frameworks such as pytest, cucumber, etc on big data platforms
  • Knowledge of GraphQL, Cypher & DjangoCandidate should have the required knowledge to implement the required tasks, test it, deploy it to different environments, and setup observability to view how the API/pipeline is behaving.Qualifications
    • Bachelor's or Master's degree in Computer Science or a similar field of study
    • Extensive data engineering experience is required.
    • 7+ years of hands-on work in the field of software development.
    • 3+ years of experience with data applications and tools (Snowflake, Databricks, Spark, EMR, Presto, Datadog, DBT, etc.)
    • 3+ years of experience in designing and building ETL pipelines to process data at scale
    • 3+ years of hands-on experience with AWS services like S3, EC2, ECS, IAM, CloudWatch, Lambda, and other AWS services
    • 3+ years of experience with CI/CD tools such as Jenkins or AWS CodeBuild, Terraform, etc.
    • Must be versatile and an inquisitive problem solver with excellent communication and writing skills.
    • Strong skills in SQL and proven experience working with at least one of the major database platforms.Employees at all levels are expected to:
      • Understand our Operating Principles; make them the guidelines for how you do your job.
      • Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services.
      • Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences.
      • Win as a team - make big things happen by working together and being open to new ideas.
      • Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs and helping us elevate opportunities to do better for our customers.
      • Drive results and growth.
      • Respect and promote inclusion & diversity.
      • Do what's right for each other, our customers, investors and our communities.Disclaimer:
        • This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.About Our Perks & Benefits:We are determined to create an environment where our employees feel valued, understand our business goals, and are motivated.Here's a look at just some of the perks and benefits we make available to our US-based employees:
          • Medical & Dental
          • 401(k) Savings Plan
          • Generous paid time off
          • Life Milestones - from adoption assistance, childcare resources, pet insurance, and more, Comcast supports you at all life stages.
          • Courtesy Services - We offer all of our full-time employees in serviceable areas free digital TV and internet.
          • Discount tickets for Universal Resorts, including theme park tickets and onsite hotel rooms.Learn more at Reasonable AccommodationWe will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, perform essential job functions, and receive other benefits and privileges of employment. Please contact us to request an accommodation.Disclaimer:
            • This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.Comcast is proud to be an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the on our careers site for more details.EducationBachelor's DegreeWhile possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.Relevant Work Experience10 Years +

Keywords: Comcast Corporation, Hamilton , Principal Data Engineer, Engineering , Philadelphia, New Jersey