Data Engineer Lead with Spark 96 views

Hi ,

please go through the below Job Description and let me know your interest….

Title: Data Engineer Lead with Spark

Location: Cambridge MA

Duration: 12+

EXP Required: 10+

Look for candidate having very good experience in Spark

Actual Requirement

Required Skills:

• Must have minimum of 10 years of overall IT experience with at least 6 years of Big Data (Hadoop, Spark, AWS) Administration experience. (Developers 5+ years)
• Must have experience with Big Data tool sets: Hadoop, Spark, data bricks, cloud (AWS)
• Must have 2+ years of spark experience.
• Previous Databricks experience is required.
• Good to have knowledge on Mathematica, MATLAB, Stella.
• Hands-on experience with installation, configuration, and support of Hadoop, spark, cloud integrations (AWS).
•  Experience with Data science tools R, Jupyter
• Experience with UNIX experience including scripting
• Must have Experience with at least one Graph Technology like Neo4J, Tiger Graph, etc.
• Hands on experience on Cloud Computing, Amazon Web Services, Azure.
•   Knowledge on monitoring and auditing AWS tools like CloudWatch, CloudTrail etc.
• Proficient working with AWS security and access management
•  Good analytical and problem-solving skills.
•  Strong communication skills and able to work independently as well as with the team.

Core Requirements

• Databricks is a combination of advanced spark cluster and a recent data lake framework called “Delta Lake “.