Senior Data Engineer with (AWS )
Senior Data Engineer with (AWS )
Company: R Systems International Ltd.
Experience: from 5 to 8 year
Location: Noida ( Sector-59 Noida)
Job Description:
Roles and Responsibilities
- Bachelors degree in Computer Science, Engineering, or a related technical field, Masters degree an asset
- A minimum of 5 years of Functional Programming and/or Object Oriented Programming (OOP) experience
- A minimum of 3 years experience writing and optimizing SQL queries
- A minimum of 3 years experience with Apache Spark for big data processing
- Extensive knowledge of Python programming language and its data manipulation libraries (Pandas and Numpy)
- Expertise in data modeling and an advanced understanding of data architecture
- Expertise with RDBMS and NoSQL databases at scale
- Experience with Apache Airflow or other similar data pipelining and workflow scheduling framework (Luigi, Azkaban)
- Ability to use containers, orchestration frameworks, and other DevOps tools (Kubernetes, Terraform, Giant Swarm, etc.)
- Proficiency with cloud resources (AWS/Google Cloud/Azure) with the ability to operate them for the
- components owned. Certification is an asset.
- Knowledge of the AWS services (Redshift, Glue, Athena, S3, etc.) an asset
- Knowledge of big data technology (Databricks, Hadoop, Hive, Pig, Presto) an asset
- Familiarity with continuous integration and automated pipeline tools (Jenkins, Travis, etc.)
Proficiency in Git
Keyskills:
- S3
- Pyspark
- Terraform
- Spark
- ETLAws Glue
- Amazon Web services
- Athena
- Numpy
- Python