38,620 Cloud Data Engineering jobs in India

Job Summary:

We are seeking a highly skilled Cloud Data Engineering Expert to design, develop, and deploy end-to-end data pipelines on AWS cloud infrastructure.

• Responsibilities:
• Design and build large-scale data processing pipelines in a production environment using AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.
• Implement data processing and transformation workflows using Apache Spark and SQL to support analytics and reporting requirements.
• Build and maintain orchestration workflows to automate data pipeline execution, scheduling, and monitoring.
• Collaborate with analysts and business stakeholders to understand data requirements and deliver scalable data solutions.
• Optimize data pipelines for performance, reliability, and cost-effectiveness by leveraging AWS best practices and cloud-native technologies.

Qualifications:

• Minimum 8+ years of experience building and deploying large-scale data processing pipelines in a production environment.
• Hands-on experience in designing and building data pipelines on AWS cloud infrastructure.
• Strong proficiency in AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.
• Strong experience with Apache Spark for data processing and analytics.
• Hands-on experience in orchestrating and scheduling data pipelines using AppFlow, Event Bridge, and Lambda.
• Solid understanding of data modeling, database design principles, and SQL and Spark SQL.

We are seeking a skilled Cloud Data Engineer to join our team.

• Key Responsibilities:
• Design, develop, and maintain scalable data pipelines using cloud-native tools like AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow.
• Architect and implement data lakes and data warehouses on cloud platforms like AWS, Azure, GCP.
• Develop and optimize data ingestion, transformation, and loading processes using Databricks, Snowflake, Redshift, BigQuery, Azure Synapse.
• Implement data integration processes using tools like Informatica, SAP Data Intelligence.
• Data Integration and Management:
• Integrate various data sources, including relational databases, APIs, unstructured data, and ERP systems into the data lake.
• Ensure data quality and integrity through rigorous testing and validation.
• Performance Optimization:
• Monitor and optimize the performance of data pipelines and data integration processes.
• Collaboration and Communication:
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Documentation and Maintenance:
• Document technical solutions, processes, and workflows.
• Maintain and troubleshoot existing data pipelines and data integrations.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.

Experience:

• 7+ years of experience as a Data Engineer or in a similar role.
• Proven experience with cloud platforms: AWS, Azure, and GCP.
• Hands-on experience with cloud-native data integration tools like AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow.

Skills:

• Strong programming skills in Python, Java, or Scala.
• Proficient in SQL and query optimization techniques.

Benefits:

• Remote work options.
• Flexible working hours.
• Opportunities for professional growth and development.

About Us:

• We offer a collaborative team environment and state-of-the-art technology.

We are seeking a skilled Data Engineer to join our team and drive the design, development, and maintenance of complex data assets and products. This role involves collaborating with various stakeholders to understand data requirements and create effective solutions.

The ideal candidate will have strong knowledge of Python and PySpark, as well as proficiency in SQL, Hadoop, Hive, Azure, Databricks, and Greenplum. Experience in writing SQL queries to retrieve metadata and tables from various data management systems is also highly valued.

Key Responsibilities:

1. Design, develop, and maintain ETL processes that meet business requirements.
2. Create detailed documentation of data integration and diagram processes.
3. Lead data validation, UAT, and regression testing for new data asset creation.
4. Develop and maintain data models, including schema design and optimization.
5. Design and manage data pipelines that automate data flow, ensuring data quality and consistency.

• Strong programming skills in Python, PySpark, and SQL.
• Proficiency in data engineering tools such as Azure Data Factory, Azure Databricks, and Hadoop.
• Experience in big data technologies like Spark and distributed computing frameworks.
• Excellent communication and collaboration skills when working with stakeholders and business teams.
• Strong problem-solving and troubleshooting abilities.
• Ability to establish comprehensive data quality test cases and procedures.

As a member of our team, you will have the opportunity to work on challenging projects, collaborate with experienced professionals, and contribute to the growth and success of our organization.

We offer a competitive salary and benefits package, as well as opportunities for professional development and advancement.

We are currently seeking experienced professionals to join our global data engineering team. Their primary responsibilities will include onboarding new data sources, designing and building cloud-based data ingest pipelines, developing custom plugins, handling incremental and full data loading strategies, performing data enrichment, standardization, cleansing, aggregation, and carrying out data operations activities.

Some key tasks include negotiating IT data contracts, defining effective data validation rules, error scenarios, and retry mechanisms. They must also ensure data integrity, consistency, and compliance with business standards. Additionally, they will review and refine technical requirements, implement data migration procedures, and protect solution data masking and lineage capabilities as needed.

This role requires a deep understanding of data management principles, including data warehousing, ETL processes, and data governance. The ideal candidate will have experience working with cloud-based platforms, such as GCP, and be proficient in programming languages like Python or Java.

To succeed in this position, you should possess excellent problem-solving skills, be able to work independently, and have strong communication and collaboration abilities. You will be part of a dynamic team that values innovation, teamwork, and continuous learning.