2,784 Etl Processes jobs in India
Deputy Manager – Data Engineering - ETL Developer
Posted today
Job Viewed
Job Description
We are seeking an experienced and driven Deputy Manager – Data Engineering to lead data warehouse development, manage high-volume data pipelines, and drive the implementation of our data strategy. The role involves close collaboration with business stakeholders, system owners, and cross-functional teams to translate business requirements into scalable and efficient data solutions.
Key Responsibilities:
- Coordinate with business teams and system owners to design and implement enterprise data strategies
- Translate business requirements into data models and ETL designs
- Manage petabyte-scale data systems for product development and real-time analytics
- Design and maintain robust, scalable data pipelines
- Develop and maintain CI/CD pipelines for data engineering solutions on enterprise data platforms (on-prem)
- Define data ingestion processes for structured and unstructured data
- Ensure data quality, metadata management, and transformation standards are followed
- Implement and advocate for data engineering best practices
- Lead and mentor team members, providing technical guidance and supporting talent development
- Strive for operational excellence and high-impact data delivery
Requirements
Key Technical Skills:
- Data warehouse design
- ETL development
- Data pipeline creation and automation
- Structured and unstructured data handling
- Big data technologies and large-scale data processing
- Real-time data stream processing
- Data ingestion and distribution techniques
- Working with both relational and non-relational databases
- Cloud data platform experience (optional)
- Strong communication and collaboration skills
Benefits
This is an excellent opportunity to lead enterprise-level data initiatives and contribute to innovation in the financial domain. If you're passionate about high-performance data systems and team leadership, we'd love to hear from you.
Deputy Manager – Data Engineering - ETL Developer
Posted today
Job Viewed
Job Description
We are seeking an experienced and driven Deputy Manager – Data Engineering to lead data warehouse development, manage high-volume data pipelines, and drive the implementation of our data strategy. The role involves close collaboration with business stakeholders, system owners, and cross-functional teams to translate business requirements into scalable and efficient data solutions.
Key Responsibilities:
- Coordinate with business teams and system owners to design and implement enterprise data strategies
- Translate business requirements into data models and ETL designs
- Manage petabyte-scale data systems for product development and real-time analytics
- Design and maintain robust, scalable data pipelines
- Develop and maintain CI/CD pipelines for data engineering solutions on enterprise data platforms (on-prem)
- Define data ingestion processes for structured and unstructured data
- Ensure data quality, metadata management, and transformation standards are followed
- Implement and advocate for data engineering best practices
- Lead and mentor team members, providing technical guidance and supporting talent development
- Strive for operational excellence and high-impact data delivery
Requirements
Key Technical Skills:
- Data warehouse design
- ETL development
- Data pipeline creation and automation
- Structured and unstructured data handling
- Big data technologies and large-scale data processing
- Real-time data stream processing
- Data ingestion and distribution techniques
- Working with both relational and non-relational databases
- Cloud data platform experience (optional)
- Strong communication and collaboration skills
Benefits
This is an excellent opportunity to lead enterprise-level data initiatives and contribute to innovation in the financial domain. If you're passionate about high-performance data systems and team leadership, we'd love to hear from you.
Requirements
Sitecore 10, Sitecore JSS, Headless CMS, JSS Layout Services, ReactJs Code , CI/CD Pipeline , Headless/Integrated Flow, Experience in Frontend and Content Hub
Deputy Manager – Data Engineering - ETL Developer
Posted 18 days ago
Job Viewed
Job Description
Data Engineering
Posted today
Job Viewed
Job Description
Job Title: Middleware Engineer
Position: Data Engineer
Experience: 5-6yrs
Category: IT Infrastructure
Main location: India, Karnataka, Bangalore
Employment Type: Full Time
Qualification: Bachelor's degree in Computer Science or related field or higher.
Roles and Responsibilities
Data Engineer - 5-6 years experience.
Responsibilities
===
Design, develop, and maintain data architectures, pipelines, and workflows for the collection, processing, storage, and retrieval of large volumes of structured and unstructured data from multiple sources.
Collaborate with cross-functional teams to identify and prioritize data engineering requirements and to develop and deploy data-driven solutions to address business challenges.
Build and maintain scalable data storage and retrieval systems (e.g., data lakes, data warehouses, databases), fault-tolerant, and high-performance data platforms on cloud infrastructure such as AWS, Azure, or Google Cloud Platform.
Develop and maintain ETL workflows, data pipelines, and data transformation processes to prepare data for machine learning and AI applications.
Implement and optimize distributed computing frameworks such as Hadoop, Spark, or Flink to support high-performance and scalable processing of large data sets.
Build and maintain monitoring, alerting, and logging systems to ensure the availability, reliability, and performance of data pipelines and data platforms.
Collaborate with Data Scientists and Machine Learning Engineers to deploy models on production environments and ensure their scalability, reliability, and accuracy.
Requirements:
===
Bachelor s or master s degree in computer science, engineering, or related field.
At least 5-6 years of experience in data engineering, with a strong background in machine learning, cloud computing and big data technologies.
Experience with at least one major cloud platform (AWS, Azure, GCP).
Proficiency in programming languages like Python, Java, and SQL.
Experience with distributed computing technologies such as Hadoop, Spark, and Kafka.
Familiarity with database technologies such as SQL, NoSQL, NewSQL.
Experience with data warehousing and ETL tools such as Redshift, Snowflake, or Airflow.
Strong problem-solving and analytical skills.
Excellent communication and teamwork skills.
Preferred qualification:
===
Experience with DevOps practices and tools such as Docker, Kubernetes, or Ansible, Terraform.
Experience with data visualization tools such as Tableau, Superset, Power BI, or Plotly, D3.js.
Experience with stream processing frameworks such as Kafka, Pulsar or Kinesis.
Experience with data governance, data security, and compliance.
Experience with software engineering best practices and methodologies such as Agile or Scrum.
Must Have Skills
===
data engineer with expertise in machine learning, cloud computing , and big data technologies.
Data Engineering Experince on multiple clouds one of them , preferably GCP
data lakes, data warehouses, databases
ETL workflows, data pipelines,data platforms
Hadoop, Spark, or Flink
Hadoop, Spark, and Kafka
SQL, NoSQL, NewSQL
Redshift, Snowflake, or Airflow
Senior Data Engineering Analyst - Data Engineering
Posted 2 days ago
Job Viewed
Job Description
**Primary Responsibilities:**
+ Ensures that all the standard requirements have been met and is involved in performing the technical analysis
+ Assisting the project manager by compiling information from the current systems, analyzing the program requirements, and ensuring that it meets the specified time requirements
+ Resolves moderate problems associated with the designed programs and provides technical guidance on complex programming
+ Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
**Required Qualifications:**
+ Graduate degree or equivalent experience
+ 4+ years of experience in database architecture, engineering, design, optimization, security, and administration; as well as data modeling, big data development, Extract, Transform, and Load (ETL) development, storage engineering, data warehousing, data provisioning
+ Good hands-on experience in Azure, ADF and Databricks
+ Experience in RDBMS like Oracle, SQL Server, Oracle, DB2 etc.
+ Knowledge of Azure, ADF, Databricks, Scala, Airflow, DWH concepts
+ Good knowledge in Unix and shell scripting
+ Good understanding of Extract, Transform, and Load (ETL) Architecture, Cloud, Spark
+ Good understanding of Data Architecture
+ Understanding of the business needs and designing programs and systems that match the complex business requirements and records all the specifications that are involved in the development and coding process
+ Understanding of QA and testing automation process
+ Proven ability to participate in agile development projects for batch and real time data ingestion
+ Proven ability to work with business and peers to define, estimate, and deliver functionality
+ Proven ability to be involved in creating proper technical documentation in the work assignments
**Preferred Qualifications:**
+ Knowledge of Agile, Automation, Big Data, DevOps, Python, - Scala/Pyspark programming, HDFS, Hive, Python, AI/ML
+ Understanding and knowledge of Agile
_At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission._
#Nic
Senior Data Engineering Analyst - Data Engineering
Posted today
Job Viewed
Job Description
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.
Primary Responsibilities:
Required Qualifications:
Preferred Qualifications:
At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone–of every race, gender, sexuality, age, location and income–deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.
#Nic
Mgr Data Engineering
Posted 1 day ago
Job Viewed
Job Description
Amgen harnesses the best of biology and technology to fight the world's toughest diseases, and make people's lives easier, fuller and longer. We discover, develop, manufacture and deliver innovative medicines to help millions of patients. Amgen helped establish the biotechnology industry more than 40 years ago and remains on the cutting-edge of innovation, using technology and human genetic data to push beyond what's known today.
ABOUT THE ROLE
We are seeking a seasoned Senior Data Engineer to drive the development and implementation of our data strategy. The ideal candidate possesses a strong blend of technical expertise and data-driven problem-solving skills. As a Senior Data Engineer, you will play a crucial role in designing, building, and optimizing our data pipelines and platforms while mentoring junior engineers.
Roles & Responsibilities:
+ Contribute to the design, development, and implementation of automations to monitor our data and cloud platforms for reliability, cost, and maintenance.
+ Take ownership of platform management, managing scope, timelines, and risks.
+ Ensure data quality and integrity through rigorous testing and monitoring.
+ Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions.
+ Work closely with data analysts, data scientists, and business stakeholders to understand data requirements.
+ Identify and resolve complex data-related challenges.
+ Adhere to data engineering best practices and standards.
+ Stay up-to-date with the latest data technologies and trends.
+ Execute POCs on new technologies to advance the capabilities and deliveries in our data strategy.
**Basic Qualifications**
+ Doctorate degree / Master's degree / Bachelor's degree and 8 to 13 years (in Computer Science
**Functional Skills:**
**Must-Have Skills:**
+ Demonstrated hands-on experience with cloud platforms (AWS, Azure, GCP) and the ability to architect cost-effective and scalable data solutions.
+ Proficiency in Python, PySpark, SQL. Hands-on experience with big data ETL performance tuning.
+ Strong analytical and problem-solving skills to address complex data challenges.
+ Manage and administered cloud-based environments such as (AWS, Azure, GCP) and data platforms (Databricks, Snowflake)
+ Experiencing working with troubleshooting cloud environment issues (networking, virtual machines)
+ Effective communication and interpersonal skills to collaborate with cross-functional teams.
**Good-to-Have Skills:**
+ Experienced with data modeling and performance tuning for both OLAP and OLTP databases
+ Experience with administering lifecycle and platform management policies on usage of cloud resources
+ Experienced working with Apache Spark, Apache Airflow
+ Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops
+ Experienced with AWS, GCP or Azure cloud services
**EQUAL OPPORTUNITY STATEMENT**
Amgen is an Equal Opportunity employer and will consider you without regard to your race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or disability status.
We will ensure that individuals with disabilities are provided with reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request an accommodation.
Be The First To Know
About the latest Etl processes Jobs in India !
Data Engineering Consultant
Posted 2 days ago
Job Viewed
Job Description
Functions may include database architecture, engineering, design, optimization, security, and administration; as well as data modeling, big data development, Extract, Transform, and Load (ETL) development, storage engineering, data warehousing, data provisioning and other similar roles. Responsibilities may include Platform-as-a-Service and Cloud solution with a focus on data stores and associated eco systems. Duties may include management of design services, providing sizing and configuration assistance, ensuring strict data quality, and performing needs assessments. Analyzes current business practices, processes and procedures as well as identifying future business opportunities for leveraging data storage and retrieval system capabilities. Manages relationships with software and hardware vendors to understand the potential architectural impact of different vendor strategies and data acquisition. May design schemas, write SQL or other data markup scripting and helps to support development of Analytics and Applications that build on top of data. Selects, develops and evaluates personnel to ensure the efficient operation of the function.
**Primary Responsibilities:**
+ Build constructive partnerships with business leadership to proactively recommend solutions to business issues through practical, elegant, and effective business intelligence and reporting vehicles
+ Direct deployment of enterprise data strategy vision through successful program delivery
+ Collaborate with global and contractor leaders to develop partnerships that allow for capacity flexibility and offer equivalent and or superior quality delivery capability
+ Effectively manages a balanced scorecard metric set that would include key measures such as quality & timely delivery.
+ Incorporate best practices and standards in metadata and report/dashboard development
+ Translates highly complex concepts in ways that can be understood by a variety of audiences
+ Function as a mentor for other team members, providing assistance where needed
+ Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
**Required Qualifications:**
+ Undergraduate degree or equivalent experience
+ 5+years of experience communicating with senior level business leaders and stakeholders
+ 5+years of experience with relational database architecture, design and development
+ 3+years of experience working with ETL tools or Advanced SQL
+ 3+years of experience working on or managing Data Warehousing projects
+ 1+years of successive project management or technical lead experience with a focus on reporting and analytics and/or management of global teams
**Preferred Qualifications:**
+ Database development experience in Snowflake
+ Good experience with ADF, SSIS and python
_At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission._
Director, Data Engineering
Posted 3 days ago
Job Viewed
Job Description
**Essential Job Responsibilities:**
+ Participate in formulating Parexel Data strategy and business system architecture plans, assessing the cost and feasibility of system requests, and ensuring data services support strategic and near-term needs.
+ Will manage the framework and drive data services by designing and implementing a strategy supporting asset standardization and reuse.
+ Act as a general expert within the Data organization to develop appropriate support for projects from various technology and business teams.
+ Maintain compliance with company SOPs relating system validation and GxP standards.
+ Create a high-performance and collaborative team environment.
+ Ability to lead dynamically and energize multi-disciplined work teams to learn and apply new skills/techniques to respond to business needs.
+ Strong verbal and written communication skills.
+ Strong analytical, problem-solving, and conceptual skills.
+ Strong teamwork and interpersonal skills; ability to communicate and persuade at all management levels and thrive in a cross-functional environment.
+ Knowledge of the company's business practices and direction, as well as familiarity with the company's products and resources.
+ Knowledge and experience in developing strategic information systems plans.
**Technical Skills:**
+ Significant experience in implementing and supporting enterprise services applications using cloud platforms.
+ Strong experience and knowledge of modern data management and sharing tools, programming languages, relational databases, data transformation, and development frameworks.
+ Experience with GxP in Lifescience setting is mandatory.
+ Proven experience with data and cloud platforms - Azure, Databricks, Snowflake, and Denodo preferred.
+ Deep understanding of architectural recommendations and application development designs and estimations.
+ Significant experience implementing and supporting medallion architecture-based Data Lake and Power BI are strongly preferred.
+ Deep understanding of Familiarity with application architectures and enterprise application ecosystems.
**Job experience:**
+ 15+ years of IT experience, including developing, implementing, and maintaining large-scale enterprise integration services tools and platforms across multiple hardware and software platforms; significant expertise and knowledge in new computing architectures.
+ 8+ years of experience leading teams in data delivery management and execution involving multiple major development projects in data management, data integration, and analytics delivery.
+ 5+ years of management experience in which one or more of the following has occurred:
+ Successful development and implementation of applications in new computing architecture environments using new and emerging technologies.
+ A track record of managing technical staff through various life-cycle phases (cross-functional experience highly desirable).
+ Experience managing financial resources to meet goals and multiple projects.
+ Developed efficient and effective solutions to diverse and complex business problems.
+ Clinical IT experience preferred.
**Education:**
+ Bachelor's degree in computer science, Information Systems, Business Administration, or other related field (or equivalent work experience).
+ Master's degree preferred.
#LI-REMOTE
EEO Disclaimer
Parexel is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to legally protected status, which in the US includes race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.
Data Engineering Analyst
Posted 5 days ago
Job Viewed
Job Description
We are seeking a talented and motivated Data Engineer to join our growing data team. You will play a key role in building scalable data pipelines, optimizing data infrastructure, and enabling data-driven solutions.
**Primary Responsibilities:**
+ Design, develop, and maintain scalable ETL/ELT pipelines for batch and real-time data processing
+ Build and optimize data models and data warehouses to support analytics and reporting
+ Collaborate with analysts and software engineers to deliver high-quality data solutions
+ Ensure data quality, integrity, and security across all systems
+ Monitor and troubleshoot data pipelines and infrastructure for performance and reliability
+ Contribute to internal tools and frameworks to improve data engineering workflows
+ Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
**Required Qualifications:**
+ 5+ years of experience working on commercially available software and / or healthcare platforms as a Data Engineer
+ 3+ years of solid experience designing and building Enterprise Data solutions on cloud
+ 1+ years of experience developing solutions hosted within public cloud providers such as Azure or AWS or private cloud/container-based systems using Kubernetes/OpenShift
+ Experience with some of the modern relational databases
+ Experience with Data warehousing services preferably Snowflake
+ Experience in using modern software engineering and product development tools including Agile / SAFE, Continuous Integration, Continuous Delivery, DevOps etc.
+ Solid experience of operating in a quickly changing environment and driving technological innovation to meet business requirements
+ Skilled at optimizing SQL statements
+ Subject matter expert on Cloud technologies preferably Azure and Big Data ecosystem
**Preferred Qualifications:**
+ Experience with real-time data streaming and event-driven architectures
+ Experience building Big Data solutions on public cloud (Azure)
+ Experience building data pipelines on Azure with skills Databricks spark, scala, Azure Data factory, Kafka and Kafka Streams, App services, Az Functions
+ Experience developing RESTful Services in .NET, Java or any other language
+ Experience with DevOps in Data engineering
+ Experience with Microservices architecture
+ Exposure to DevOps practices and infrastructure-as-code (e.g., Terraform, Docker)
+ Knowledge of data governance and data lineage tools
+ Ability to establish repeatable processes, best practices and implement version control software in a Cloud team environment
_At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission._