27,086 Lead Data Engineer jobs in India

Lead Big Data Engineer

Pune, Maharashtra Qualys

Posted today

Job Viewed

Tap Again To Close

Job Description

Description

:

We are seeking a talented Lead Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.

Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product.

This is a fantastic opportunity to be an integral part of a team building Qualys next generation platform using Big Data & Micro-Services based technology to process over billions of transactions data per day, leverage open-source technologies, and work on challenging and business-impacting initiatives.

Responsibilities:

  • Be the thought leader in data platform and pipeline along with Risk Evaluation.

  • Provide technical leadership to the engineering organization on data platform design, roll out and evolution.

  • Liason to product teams, professional services and sales engineers on solution and trade-off reviews and represent engineering in such conversations.

  • Drive technology explorations and roadmaps.

  • Serve as a technical lead on our most demanding, cross-functional departments.

  • Ensure the quality of architecture and design of systems.

  • Functionally decompose complex problems into simple, straight-forward solutions.

  • Fully and completely understand system interdependencies and limitations.

  • Possess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.

  • Leverage knowledge of internal and industry prior art in design decisions.

  • Effectively research and benchmark cloud technology against other competing systems in the industry.

  • Able to document the details so it will be easy for developers to understand the requirements.

  • Assisting developers with proper requirements and directions.

  • Assist in the career development of others, actively mentoring individuals and the community on advanced technical issues and helping managers guide the career growth of their team members.

  • Exert technical influence over multiple teams, increasing their productivity and effectiveness by sharing your deep knowledge and experience.

  • Able to share knowledge and train others.

  • Qualifications:

  • Bachelor’s degree in computer science or equivalent

  • 8+ years of total experience.

  • 4+ years of relevant experience in design and architecture Big Data solutions using Spark

  • 3+ years experience in working with engineering resources for innovation.

  • 4+ years experience in understanding Big Data events flow pipeline.

  • 3+ years experience in performance testing for large infrastructure.

  • 3+ In depth experience in understanding various search solutions solr/elastic.

  • 3+ years experience in Kafka

  • In depth experience in Data lakes and related ecosystems.

  • In depth experience of messing queue 

  • In depth experience in giving requirements to build a scalable architecture for Big data and Micro-services environments.

  • In depth experience in understanding caching components or services 

  • Knowledge in Presto technology.

  • Knowledge in Airflow.

  • Hands-on experience in scripting and automation

  • In depth understanding of RDBMS/NoSQL, Oracle ,Cassandra ,Kafka ,Redis, Hadoop, lambda architecture, kappa , kappa + architectures with flink data streaming and rule engines

  • Experience in working with ML modelsengineering and related deployment.

  • Design and implement secure big data clusters to meet many compliances and regulatory requirements.

  • Experience in leading the delivery of large-scale systems focused on managing the infrastructure layer of the technology stack.

  • Strong experience in doing performance benchmarking testing for Big data technologies.

  • Strong troubleshooting skills.

  • Experience leading development life cycle process and best practices

  • Experience in Big Data services administration would be added value.

  • Experience with Agile Management (SCRUM, RUP, XP), OO Modeling, working on internet, UNIX, Middleware, and database related projects.

  • Experience mentoring/training the engineering community on complex technical issue.

  • Project management experience

  • This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Chennai, Tamil Nadu Huron Consulting Group

    Posted 4 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Huron helps its clients drive growth, enhance performance and sustain leadership in the markets they serve. We help healthcare organizations build innovation capabilities and accelerate key growth initiatives, enabling organizations to own the future, instead of being disrupted by it. Together, we empower clients to create sustainable growth, optimize internal processes and deliver better consumer outcomes.
    Health systems, hospitals and medical clinics are under immense pressure to improve clinical outcomes and reduce the cost of providing patient care. Investing in new partnerships, clinical services and technology is not enough to create meaningful and substantive change. To succeed long-term, healthcare organizations must empower leaders, clinicians, employees, affiliates and communities to build cultures that foster innovation to achieve the best outcomes for patients.
    Joining the Huron team means you'll help our clients evolve and adapt to the rapidly changing healthcare environment and optimize existing business operations, improve clinical outcomes, create a more consumer-centric healthcare experience, and drive physician, patient and employee engagement across the enterprise.
    Join our team as the expert you are now and create your future.
    As a Lead Data Engineer, you will play a key role in supporting our various business objectives. You will collaborate directly with all members of the Engineering team and key individuals from other business units, lead the requirements discovery process, lead interactive prototyping sessions, build required data models, and prepare the models for release to production. You will utilize several different software tools and architectures to accomplish these objectives. The successful candidate will demonstrate the ability to consistently engineer data-driven models and solutions that solve problems and identify opportunities to drive operational excellence. Responsibilities Include: - Interact with key business unit leaders to elicit requirements and develop prototypes. - Successfully translate business requirements into analytical model specifications. - Identify opportunities to improve business outcomes with new data-driven solutions. - Design and coordinate the development of data-driven processes, analyses, and tools. - Work with the database architect to plan appropriate data structures to support models. - Build frameworks for automating operational models for continual analysis and insight. - Present concise and clear recommendations to business unit and practice leadership, including properly design visualizations, where appropriate. - Design and execute proper experiments to improve operations. - Assist with developing scripts and applications necessary to support data model implementation.
    **Duties & Responsibilities:**
    You will be responsible to handle the core data aspects of software engineering and data science; use software engineering principles to develop algorithms that automate the data flow process. Also collaborate with data scientists to build machine learning and analytics infrastructure, from testing to deployment.
    + Building and maintaining data infrastructure for the optimal extraction, transformation, and loading of data from a wide variety of sources such as AWS data lakes and data warehouse.
    + Ensuring data accessibility at all times and implementing company data policies with respect to data privacy and confidentiality
    + Cleaning and wrangling data from primary and secondary sources into formats that can be efficiently utilized by data scientists and other data consumers
    + Collaborating with engineering teams, data scientists, and other stakeholders to understand how data can be leveraged to meet business needs
    **Job Description:**
    + Bachelor's Degree.
    + 8+ years of experience with 5+ years of strong experience working with data.
    + Strong communication skills: ability to be well-organized, clear, and concise in conversations and written communications with development team members, business analysts, and project management.
    + Experience in Healthcare will be an added advantage.
    + Very strong communication and leadership skills and works well independently
    + Proficiency in Data Engineering best practices.
    + Experience working with Agile development methodologies
    + Good knowledge of SQL and NoSQL databases and are skilled in writing SQL queries to manipulate and retrieve data.
    + Good knowledge and hands-on experience on Python, SQL, Scala, R programing language.
    + Good Data Base knowledge on MySQL and SQL Server, strong emphasis on relational database design, scripting, and stored procedures
    + Proficiency with AWS tools: S3, Athena, Glue, RDS, Redshift, Step Functions, Lake Formation, Cloud Formation, Event Bridge, Lambda
    + Experience working with structured, semi-structured and un-structured data, ETL, data cleansing, and data conversion, data parsing
    + Experience in data modeling complex data warehouse architecture in Redshift/Athena
    + Proficiency in various data cleansing techniques, tools and ETL process.
    + Fimiliar with symantic data modelling techniques using Cube.js or (SSAS/Looker).
    + Able to manage your own time and work well both independently and as part of a team.
    **Position Level**
    Senior Associate
    **Country**
    India
    At Huron, we're redefining what a consulting organization can be. We go beyond advice to deliver results that last. We inherit our client's challenges as if they were our own. We help them transform for the future. We advocate. We make a difference. And we intelligently, passionately, relentlessly do great work.together.
    Are you the kind of person who stands ready to jump in, roll up your sleeves and transform ideas into action? Then come discover Huron.
    Whether you have years of experience or come right out of college, we invite you to explore our many opportunities. Find out how you can use your talents and develop your skills to make an impact immediately. Learn about how our culture and values provide you with the kind of environment that invites new ideas and innovation. Come see how we collaborate with each other in a culture of learning, coaching, diversity and inclusion. And hear about our unwavering commitment to make a difference in partnership with our clients, shareholders, communities and colleagues.
    Huron Consulting Group offers a competitive compensation and benefits package including medical, dental, and vision coverage to employees and dependents; a 401(k) plan with a generous employer match; an employee stock purchase plan; a generous Paid Time Off policy; and paid parental leave and adoption assistance. Our Wellness Program supports employee total well-being by providing free annual health screenings and coaching, bank at work, and on-site workshops, as well as ongoing programs recognizing major events in the lives of our employees throughout the year. All benefits and programs are subject to applicable eligibility requirements.
    Huron is fully committed to providing equal employment opportunity to job applicants and employees in recruitment, hiring, employment, compensation, benefits, promotions, transfers, training, and all other terms and conditions of employment. Huron will not discriminate on the basis of age, race, color, gender, marital status, sexual orientation, gender identity, pregnancy, national origin, religion, veteran status, physical or mental disability, genetic information, creed, citizenship or any other status protected by laws or regulations in the locations where we do business. We endeavor to maintain a drug-free workplace.
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Noida, Uttar Pradesh Huron Consulting Group

    Posted 4 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Huron helps its clients drive growth, enhance performance and sustain leadership in the markets they serve. We help healthcare organizations build innovation capabilities and accelerate key growth initiatives, enabling organizations to own the future, instead of being disrupted by it. Together, we empower clients to create sustainable growth, optimize internal processes and deliver better consumer outcomes.
    Health systems, hospitals and medical clinics are under immense pressure to improve clinical outcomes and reduce the cost of providing patient care. Investing in new partnerships, clinical services and technology is not enough to create meaningful and substantive change. To succeed long-term, healthcare organizations must empower leaders, clinicians, employees, affiliates and communities to build cultures that foster innovation to achieve the best outcomes for patients.
    Joining the Huron team means you'll help our clients evolve and adapt to the rapidly changing healthcare environment and optimize existing business operations, improve clinical outcomes, create a more consumer-centric healthcare experience, and drive physician, patient and employee engagement across the enterprise.
    Join our team as the expert you are now and create your future.
    As a Lead Data Engineer, you will play a key role in supporting our various business objectives. You will collaborate directly with all members of the Engineering team and key individuals from other business units, lead the requirements discovery process, lead interactive prototyping sessions, build required data models, and prepare the models for release to production. You will utilize several different software tools and architectures to accomplish these objectives. The successful candidate will demonstrate the ability to consistently engineer data-driven models and solutions that solve problems and identify opportunities to drive operational excellence. Responsibilities Include: - Interact with key business unit leaders to elicit requirements and develop prototypes. - Successfully translate business requirements into analytical model specifications. - Identify opportunities to improve business outcomes with new data-driven solutions. - Design and coordinate the development of data-driven processes, analyses, and tools. - Work with the database architect to plan appropriate data structures to support models. - Build frameworks for automating operational models for continual analysis and insight. - Present concise and clear recommendations to business unit and practice leadership, including properly design visualizations, where appropriate. - Design and execute proper experiments to improve operations. - Assist with developing scripts and applications necessary to support data model implementation.
    **Duties & Responsibilities:**
    You will be responsible to handle the core data aspects of software engineering and data science; use software engineering principles to develop algorithms that automate the data flow process. Also collaborate with data scientists to build machine learning and analytics infrastructure, from testing to deployment.
    + Building and maintaining data infrastructure for the optimal extraction, transformation, and loading of data from a wide variety of sources such as AWS data lakes and data warehouse.
    + Ensuring data accessibility at all times and implementing company data policies with respect to data privacy and confidentiality
    + Cleaning and wrangling data from primary and secondary sources into formats that can be efficiently utilized by data scientists and other data consumers
    + Collaborating with engineering teams, data scientists, and other stakeholders to understand how data can be leveraged to meet business needs
    **Job Description:**
    + Bachelor's Degree.
    + 8+ years of experience with 5+ years of strong experience working with data.
    + Strong communication skills: ability to be well-organized, clear, and concise in conversations and written communications with development team members, business analysts, and project management.
    + Experience in Healthcare will be an added advantage.
    + Very strong communication and leadership skills and works well independently
    + Proficiency in Data Engineering best practices.
    + Experience working with Agile development methodologies
    + Good knowledge of SQL and NoSQL databases and are skilled in writing SQL queries to manipulate and retrieve data.
    + Good knowledge and hands-on experience on Python, SQL, Scala, R programing language.
    + Good Data Base knowledge on MySQL and SQL Server, strong emphasis on relational database design, scripting, and stored procedures
    + Proficiency with AWS tools: S3, Athena, Glue, RDS, Redshift, Step Functions, Lake Formation, Cloud Formation, Event Bridge, Lambda
    + Experience working with structured, semi-structured and un-structured data, ETL, data cleansing, and data conversion, data parsing
    + Experience in data modeling complex data warehouse architecture in Redshift/Athena
    + Proficiency in various data cleansing techniques, tools and ETL process.
    + Fimiliar with symantic data modelling techniques using Cube.js or (SSAS/Looker).
    + Able to manage your own time and work well both independently and as part of a team.
    **Position Level**
    Senior Associate
    **Country**
    India
    At Huron, we're redefining what a consulting organization can be. We go beyond advice to deliver results that last. We inherit our client's challenges as if they were our own. We help them transform for the future. We advocate. We make a difference. And we intelligently, passionately, relentlessly do great work.together.
    Are you the kind of person who stands ready to jump in, roll up your sleeves and transform ideas into action? Then come discover Huron.
    Whether you have years of experience or come right out of college, we invite you to explore our many opportunities. Find out how you can use your talents and develop your skills to make an impact immediately. Learn about how our culture and values provide you with the kind of environment that invites new ideas and innovation. Come see how we collaborate with each other in a culture of learning, coaching, diversity and inclusion. And hear about our unwavering commitment to make a difference in partnership with our clients, shareholders, communities and colleagues.
    Huron Consulting Group offers a competitive compensation and benefits package including medical, dental, and vision coverage to employees and dependents; a 401(k) plan with a generous employer match; an employee stock purchase plan; a generous Paid Time Off policy; and paid parental leave and adoption assistance. Our Wellness Program supports employee total well-being by providing free annual health screenings and coaching, bank at work, and on-site workshops, as well as ongoing programs recognizing major events in the lives of our employees throughout the year. All benefits and programs are subject to applicable eligibility requirements.
    Huron is fully committed to providing equal employment opportunity to job applicants and employees in recruitment, hiring, employment, compensation, benefits, promotions, transfers, training, and all other terms and conditions of employment. Huron will not discriminate on the basis of age, race, color, gender, marital status, sexual orientation, gender identity, pregnancy, national origin, religion, veteran status, physical or mental disability, genetic information, creed, citizenship or any other status protected by laws or regulations in the locations where we do business. We endeavor to maintain a drug-free workplace.
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Bangalore, Karnataka Huron Consulting Group

    Posted 4 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Huron helps its clients drive growth, enhance performance and sustain leadership in the markets they serve. We help healthcare organizations build innovation capabilities and accelerate key growth initiatives, enabling organizations to own the future, instead of being disrupted by it. Together, we empower clients to create sustainable growth, optimize internal processes and deliver better consumer outcomes.
    Health systems, hospitals and medical clinics are under immense pressure to improve clinical outcomes and reduce the cost of providing patient care. Investing in new partnerships, clinical services and technology is not enough to create meaningful and substantive change. To succeed long-term, healthcare organizations must empower leaders, clinicians, employees, affiliates and communities to build cultures that foster innovation to achieve the best outcomes for patients.
    Joining the Huron team means you'll help our clients evolve and adapt to the rapidly changing healthcare environment and optimize existing business operations, improve clinical outcomes, create a more consumer-centric healthcare experience, and drive physician, patient and employee engagement across the enterprise.
    Join our team as the expert you are now and create your future.
    As a Lead Data Engineer, you will play a key role in supporting our various business objectives. You will collaborate directly with all members of the Engineering team and key individuals from other business units, lead the requirements discovery process, lead interactive prototyping sessions, build required data models, and prepare the models for release to production. You will utilize several different software tools and architectures to accomplish these objectives. The successful candidate will demonstrate the ability to consistently engineer data-driven models and solutions that solve problems and identify opportunities to drive operational excellence. Responsibilities Include: - Interact with key business unit leaders to elicit requirements and develop prototypes. - Successfully translate business requirements into analytical model specifications. - Identify opportunities to improve business outcomes with new data-driven solutions. - Design and coordinate the development of data-driven processes, analyses, and tools. - Work with the database architect to plan appropriate data structures to support models. - Build frameworks for automating operational models for continual analysis and insight. - Present concise and clear recommendations to business unit and practice leadership, including properly design visualizations, where appropriate. - Design and execute proper experiments to improve operations. - Assist with developing scripts and applications necessary to support data model implementation.
    **Duties & Responsibilities:**
    You will be responsible to handle the core data aspects of software engineering and data science; use software engineering principles to develop algorithms that automate the data flow process. Also collaborate with data scientists to build machine learning and analytics infrastructure, from testing to deployment.
    + Building and maintaining data infrastructure for the optimal extraction, transformation, and loading of data from a wide variety of sources such as AWS data lakes and data warehouse.
    + Ensuring data accessibility at all times and implementing company data policies with respect to data privacy and confidentiality
    + Cleaning and wrangling data from primary and secondary sources into formats that can be efficiently utilized by data scientists and other data consumers
    + Collaborating with engineering teams, data scientists, and other stakeholders to understand how data can be leveraged to meet business needs
    **Job Description:**
    + Bachelor's Degree.
    + 8+ years of experience with 5+ years of strong experience working with data.
    + Strong communication skills: ability to be well-organized, clear, and concise in conversations and written communications with development team members, business analysts, and project management.
    + Experience in Healthcare will be an added advantage.
    + Very strong communication and leadership skills and works well independently
    + Proficiency in Data Engineering best practices.
    + Experience working with Agile development methodologies
    + Good knowledge of SQL and NoSQL databases and are skilled in writing SQL queries to manipulate and retrieve data.
    + Good knowledge and hands-on experience on Python, SQL, Scala, R programing language.
    + Good Data Base knowledge on MySQL and SQL Server, strong emphasis on relational database design, scripting, and stored procedures
    + Proficiency with AWS tools: S3, Athena, Glue, RDS, Redshift, Step Functions, Lake Formation, Cloud Formation, Event Bridge, Lambda
    + Experience working with structured, semi-structured and un-structured data, ETL, data cleansing, and data conversion, data parsing
    + Experience in data modeling complex data warehouse architecture in Redshift/Athena
    + Proficiency in various data cleansing techniques, tools and ETL process.
    + Fimiliar with symantic data modelling techniques using Cube.js or (SSAS/Looker).
    + Able to manage your own time and work well both independently and as part of a team.
    **Position Level**
    Senior Associate
    **Country**
    India
    At Huron, we're redefining what a consulting organization can be. We go beyond advice to deliver results that last. We inherit our client's challenges as if they were our own. We help them transform for the future. We advocate. We make a difference. And we intelligently, passionately, relentlessly do great work.together.
    Are you the kind of person who stands ready to jump in, roll up your sleeves and transform ideas into action? Then come discover Huron.
    Whether you have years of experience or come right out of college, we invite you to explore our many opportunities. Find out how you can use your talents and develop your skills to make an impact immediately. Learn about how our culture and values provide you with the kind of environment that invites new ideas and innovation. Come see how we collaborate with each other in a culture of learning, coaching, diversity and inclusion. And hear about our unwavering commitment to make a difference in partnership with our clients, shareholders, communities and colleagues.
    Huron Consulting Group offers a competitive compensation and benefits package including medical, dental, and vision coverage to employees and dependents; a 401(k) plan with a generous employer match; an employee stock purchase plan; a generous Paid Time Off policy; and paid parental leave and adoption assistance. Our Wellness Program supports employee total well-being by providing free annual health screenings and coaching, bank at work, and on-site workshops, as well as ongoing programs recognizing major events in the lives of our employees throughout the year. All benefits and programs are subject to applicable eligibility requirements.
    Huron is fully committed to providing equal employment opportunity to job applicants and employees in recruitment, hiring, employment, compensation, benefits, promotions, transfers, training, and all other terms and conditions of employment. Huron will not discriminate on the basis of age, race, color, gender, marital status, sexual orientation, gender identity, pregnancy, national origin, religion, veteran status, physical or mental disability, genetic information, creed, citizenship or any other status protected by laws or regulations in the locations where we do business. We endeavor to maintain a drug-free workplace.
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Bangalore, Karnataka Target

    Posted 5 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    **About us:**
    Working at Target means helping all families discover the joy of everyday life. We bring that vision to life through our values and culture. Learn more about Target here ( .
    **About the Role:**
    As a **Lead Data Engineer** , you will serve as the **technical anchor** for the engineering team, responsible for designing and developing **scalable, high-performance data solutions** . You will **own and drive data architecture** that supports both functional and non-functional business needs, ensuring **reliability, efficiency, and scalability** .
    Your expertise in **big data technologies, distributed systems, and cloud platforms** will help shape the engineering roadmap and best practices for **data processing, analytics, and real-time data serving** . You will play a key role in **architecting and optimizing data pipelines** using **Hadoop, Spark, Scala/Java, and cloud technologies** to support enterprise-wide data initiatives.
    Additionally, **experience with API development for serving low-latency data** and **Customer Data Platforms (CDP)** will be a strong plus.
    **Key Responsibilities:**
    + Architect and build **scalable, high-performance data pipelines** and **distributed data processing solutions** using **Hadoop, Spark, Scala/Java, and cloud platforms (AWS/GCP/Azure)** .
    + Design and implement **real-time and batch data processing solutions** , ensuring data is efficiently processed and made available for analytical and operational use.
    + **Develop APIs and data services** to expose **low-latency, high-throughput** data for downstream applications, enabling real-time decision-making.
    + Optimize and enhance **data models, workflows, and processing frameworks** to improve performance, scalability, and cost-efficiency.
    + Drive **data governance, security, and compliance** best practices.
    + Collaborate with **data scientists, product teams, and business stakeholders** to understand requirements and deliver **data-driven solutions** .
    + Lead the **design, implementation, and lifecycle management** of data services and solutions.
    + Stay up to date with **emerging technologies** and drive adoption of best practices in **big data engineering, cloud computing, and API development** .
    + Provide **technical leadership and mentorship** to engineering teams, promoting best practices in **data engineering and API design** .
    **About You:**
    + **7+ years of experience in data engineering, software development, or distributed systems.**
    + **Expertise in big data technologies** such as **Hadoop, Spark, and distributed processing frameworks.**
    + **Strong programming skills in Scala and/or Java** (Python is a plus).
    + **Experience with cloud platforms (AWS, GCP, or Azure)** and their **data ecosystem** (e.g., S3, BigQuery, Databricks, EMR, Snowflake, etc.).
    + **Proficiency in API development** using **REST, GraphQL, or gRPC** to serve real-time and batch data.
    + **Experience with real-time and streaming data architectures** (Kafka, Flink, Kinesis, etc.).
    + Strong knowledge of **data modeling, ETL pipeline design, and performance optimization** .
    + Understanding of **data governance, security, and compliance** in large-scale data environments.
    + **Experience with Customer Data Platforms (CDP) or customer-centric data processing** is a strong plus.
    + Strong problem-solving skills and ability to work in **complex, unstructured environments** .
    + Excellent communication and collaboration skills, with experience working in **cross-functional teams** .
    **Why Join Us?**
    + Work with cutting-edge **big data, API, and cloud technologies** in a fast-paced, collaborative environment.
    + Influence and shape the **future of data architecture and real-time data services** at Target.
    + Solve **high-impact business problems** using **scalable, low-latency data solutions** .
    + Be part of a culture that values **innovation, learning, and growth** .
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Hyderabad, Andhra Pradesh S&P Global

    Posted 5 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    **About the Role:**
    **Grade Level (for internal use):**
    11
    **The Role:** Lead Software Engineer
    **The Team:** The Market Intelligence Industry Data Solutions business line provides data technology and services supporting acquisition, ingestion, content management, mastering, and distribution to power our Financial Institution Group business and customer needs. We focus on platform scalability to support business operations by following a common data lifecycle that accelerates business value. Our team provides essential intelligence for the Financial Services, Real Estate, and Insurance industries.
    **The Impact:** The FIG Data Engineering team will be responsible for implementing and maintaining services and tools to support existing feed systems. This enables users to consume FIG datasets and makes FIG data available for broader consumption and processing within the company.
    **What's in it for you:** Opportunity to work with global stakeholders and engage with the latest tools and technologies.
    **Responsibilities:**
    + Build new data acquisition and transformation pipelines using advanced data processing and cloud technologies.
    + Collaborate with the broader technology team, including information architecture and data integration teams, to align pipelines with strategic initiatives.
    **What We're Looking For:**
    + Bachelor's degree in computer science or a related field, with at least 8+ years of professional software development experience.
    + Must have: **Programming languages commonly used for data processing,** **Data orchestration and workflow management systems,** **Distributed data processing framework, relational database management systems,** **B** **ig data processing frameworks**
    + Experience with large-scale data processing platforms.
    + Deep understanding of RESTful services, good API design, and object-oriented programming principles.
    + Proficiency in object-oriented or functional scripting languages.
    + Good working knowledge of relational and NoSQL databases.
    + Experience in maintaining and developing software in production environments utilizing cloud-based tools.
    + Strong collaboration and teamwork skills, along with excellent written and verbal communication abilities.
    + Self-starter and motivated individual with the ability to thrive in a fast-paced software development environment.
    + Agile experience is highly desirable.
    + Experience with data warehousing and analytics platforms will be a significant advantage.
    **What's In It For** **You?**
    **Our Purpose:**
    Progress is not a self-starter. It requires a catalyst to be set in motion. Information, imagination, people, technology-the right combination can unlock possibility and change the world.
    Our world is in transition and getting more complex by the day. We push past expected observations and seek out new levels of understanding so that we can help companies, governments and individuals make an impact on tomorrow. At S&P Global we transform data into Essential Intelligence®, pinpointing risks and opening possibilities. We Accelerate Progress.
    **Our People:**
    We're more than 35,000 strong worldwide-so we're able to understand nuances while having a broad perspective. Our team is driven by curiosity and a shared belief that Essential Intelligence can help build a more prosperous future for us all.
    From finding new ways to measure sustainability to analyzing energy transition across the supply chain to building workflow solutions that make it easy to tap into insight and apply it. We are changing the way people see things and empowering them to make an impact on the world we live in. We're committed to a more equitable future and to helping our customers find new, sustainable ways of doing business. We're constantly seeking new solutions that have progress in mind. Join us and help create the critical insights that truly make a difference.
    **Our Values:**
    **Integrity, Discovery, Partnership**
    At S&P Global, we focus on Powering Global Markets. Throughout our history, the world's leading organizations have relied on us for the Essential Intelligence they need to make confident decisions about the road ahead. We start with a foundation of **integrity** in all we do, bring a spirit of **discovery** to our work, and collaborate in close **partnership** with each other and our customers to achieve shared goals.
    **Benefits:**
    We take care of you, so you can take care of business. We care about our people. That's why we provide everything you-and your career-need to thrive at S&P Global.
    Our benefits include:
    + Health & Wellness: Health care coverage designed for the mind and body.
    + Flexible Downtime: Generous time off helps keep you energized for your time on.
    + Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
    + Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
    + Family Friendly Perks: It's not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
    + Beyond the Basics: From retail discounts to referral incentive awards-small perks can make a big difference.
    For more information on benefits by country visit: Hiring and Opportunity at S&P Global:**
    At S&P Global, we are committed to fostering a connected and engaged workplace where all individuals have access to opportunities based on their skills, experience, and contributions. Our hiring practices emphasize fairness, transparency, and merit, ensuring that we attract and retain top talent. By valuing different perspectives and promoting a culture of respect and collaboration, we drive innovation and power global markets.
    **Recruitment Fraud Alert:**
    If you receive an email from a spglobalind.com domain or any other regionally based domains, it is a scam and should be reported to . S&P Global never requires any candidate to pay money for job applications, interviews, offer letters, "pre-employment training" or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines, fraudulent domains, and how to report suspicious activity here ( .
    ---
    **Equal Opportunity Employer**
    S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, military veteran status, unemployment status, or any other status protected by law. Only electronic job submissions will be considered for employment.
    If you need an accommodation during the application process due to a disability, please send an email to:? ?and your request will be forwarded to the appropriate person?
    **US Candidates Only:** The EEO is the Law Poster ? describes discrimination protections under federal law. Pay Transparency Nondiscrimination Provision - - Professional (EEO-2 Job Categories-United States of America), IFTECH202.2 - Middle Professional Tier II (EEO Job Group), SWP Priority - Ratings - (Strategic Workforce Planning)
    **Job ID:** 318262
    **Posted On:** 2025-08-04
    **Location:** Hyderabad, Telangana, India
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Bengaluru, Karnataka Burns & McDonnell

    Posted 5 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    **Description**
    + Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.
    + Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.
    + Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.
    + Develop and enforce data engineering best practices, including coding standards, testing, and documentation.
    + Mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning and improvement.
    + Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to business operations.
    + Stay up to date with the latest industry trends and technologies, and proactively recommend improvements to our data engineering practices.
    **Qualifications**
    + Systems (MIS), Data Science or related field.
    + 15 years of experience in data engineering and/or architecture, with a focus on big data technologies.
    + Extensive production experience with Databricks, Apache Spark, and other related technologies.
    + Familiarity with orchestration and ELT tools like Airflow, dbt, etc.
    + Expert SQL knowledge.
    + Proficiency in programming languages such as Python, Scala, or Java.
    + Strong understanding of data warehousing concepts.
    + Experience with cloud platforms such as Azure, AWS, Google Cloud.
    + Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
    + Strong communication and leadership skills, with the ability to effectively mentor and guide
    + Experience with machine learning and data science workflows
    + Knowledge of data governance and security best practices
    + Certification in Databricks, Azure, Google Cloud or related technologies.
    **Job** Engineering
    **Primary Location** India-Karnataka-Bengaluru
    **Schedule:** Full-time
    **Travel:** No
    **Req ID:** 252684
    **Job Hire Type** Experienced Not Applicable #BMI N/A
    This advertiser has chosen not to accept applicants from your region.
    Be The First To Know

    About the latest Lead data engineer Jobs in India !

    Lead Data Engineer

    Mumbai, Maharashtra Burns & McDonnell

    Posted 5 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    **Description**
    + Lead the design, development, and implementation of scalable data pipelines and ELT processes using Databricks, DLT, dbt, Airflow, and other tools.
    + Collaborate with stakeholders to understand data requirements and deliver high-quality data solutions.
    + Optimize and maintain existing data pipelines to ensure data quality, reliability, and performance.
    + Develop and enforce data engineering best practices, including coding standards, testing, and documentation.
    + Mentor junior data engineers, providing technical leadership and fostering a culture of continuous learning and improvement.
    + Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to business operations.
    + Stay up to date with the latest industry trends and technologies, and proactively recommend improvements to our data engineering practices.
    **Qualifications**
    + Systems (MIS), Data Science or related field.
    + 15 years of experience in data engineering and/or architecture, with a focus on big data technologies.
    + Extensive production experience with Databricks, Apache Spark, and other related technologies.
    + Familiarity with orchestration and ELT tools like Airflow, dbt, etc.
    + Expert SQL knowledge.
    + Proficiency in programming languages such as Python, Scala, or Java.
    + Strong understanding of data warehousing concepts.
    + Experience with cloud platforms such as Azure, AWS, Google Cloud.
    + Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.
    + Strong communication and leadership skills, with the ability to effectively mentor and guide
    + Experience with machine learning and data science workflows
    + Knowledge of data governance and security best practices
    + Certification in Databricks, Azure, Google Cloud or related technologies.
    **Job** Information Technology
    **Primary Location** India-Maharashtra-Mumbai
    **Schedule:** Full-time
    **Travel:** No
    **Req ID:** 250903
    **Job Hire Type** Experienced Not Applicable #BMI N/A
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Noida, Uttar Pradesh Conduent

    Posted 5 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Through our dedicated associates, Conduent delivers mission-critical services and solutions on behalf of Fortune 100 companies and over 500 governments - creating exceptional outcomes for our clients and the millions of people who count on them. You have an opportunity to personally thrive, make a difference and be part of a culture where individuality is noticed and valued every day.
    **Job Overview**
    We are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition, transformation, loading and processing of data from a multitude of disparate data sources, including structured and unstructured data for advanced analytics and machine learning in a big data environment.
    **Responsibilities:**
    -Engineer a modern data pipeline to collect, organize, and process data from disparate sources.
    -Performs data management tasks, such as conduct data profiling, assess data quality, and write SQL queries to extract and integrate data
    - Develop efficient data collection systems and sound strategies for getting quality data from different sources
    -Consume and analyze data from the data pool to support inference, prediction and recommendation of actionable insights to support business growth.
    -Design and develop ETL processes using tools and scripting. Troubleshoot and debug ETL processes. Performance tuning and opitimization of the ETL processes.
    -Provide support to new of existing applications while recommending best practices and leading projects to implement new functionality.
    - Collaborate in design reviews and code reviews to ensure standards are met. Recommend new standards for visualizations.
    -Learn and develop new ETL techniques as required to keep up with the contemporary technologies.
    -Reviews the solution requirements and architecture to ensure selection of appropriate technology, efficient use of resources and integration of multiple systems and technology.
    -Support presentations to Customers and Partners
    -Advising on new technology trends and possible adoption to maintain competitive advantage
    **Experience Needed:**
    -8+ years of related experience is required.
    -A BS or Masters degree in Computer Science or related technical discipline is required
    -ETL experience with data integration to support data marts, extracts and reporting
    -Experience connecting to varied data sources
    -Excellent SQL coding experience with performance optimization for data queries.
    -Understands different data models like normalized, de-normalied, stars, and snowflake models. Worked with transactional, temporarl, time series, and structured and unstructured data.
    -Experience on Azure Data Factory and Azure Synapse Analytics
    -Worked in big data environments, cloud data stores, different RDBMS and OLAP solutions.
    -Experience in cloud-based ETL development processes.
    -Experience in deployment and maintenance of ETL Jobs.
    -Is familiar with the principles and practices involved in development and maintenance of software solutions and architectures and in service delivery.
    -Has strong technical background and remains evergreen with technology and industry developments.
    At least 3 years of demonstrated success in software engineering, release engineering, and/or configuration management.
    Highly skilled in scripting languages like PowerShell.
    Substantial experience in the implementation and exectuion fo CI/CD processes.
    **Additional Requirements**
    -Demonstrated ability to have successfully completed multiple, complex technical projects
    -Prior experience with application delivery using an Onshore/Offshore model
    -Experience with business processes across multiple Master data domains in a services based company
    -Demonstrates a rational and organized approach to the tasks undertaken and an awareness of the need to achieve quality.
    -Demonstrates high standards of professional behavior in dealings with clients, colleagues and staff.
    -Is able to make sound and far reaching decisions alone on major issues and to take full responsibility for them on a technical basis.
    -Strong written communication skills. Is effective and persuasive in both written and oral communication.
    - Experience with gathering end user requirements and writing technical documentation
    - Time management and multitasking skills to effectively meet deadlines under time-to-market pressure
    Conduent is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, creed, religion, ancestry, national origin, age, gender identity, gender expression, sex/gender, marital status, sexual orientation, physical or mental disability, medical condition, use of a guide dog or service animal, military/veteran status, citizenship status, basis of genetic information, or any other group protected by law.
    People with disabilities who need a reasonable accommodation to apply for or compete for employment with Conduent may request such accommodation(s) by submitting their request through this form that must be downloaded: click here to access or download the form ( . Complete the form and then email it as an attachment to . You may also click here to access Conduent's ADAAA Accommodation Policy ( .
    _At Conduent we value the health and safety of our associates, their families and our community. For US applicants while we DO NOT require vaccination for most of our jobs, we DO require that you provide us with your vaccination status, where legally permissible. Providing this information is a requirement of your employment at Conduent._
    This advertiser has chosen not to accept applicants from your region.

    Lead Data Engineer

    Velodata Global Pvt Ltd

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    Role - Lead Data Engineer

    Mandatory skillset - Big Query and Looker or Python, SQL and AWS

    Experience

    • Data Engineer (Big Query and Looker) - 8+ years (Relevant - 6+ Years)
    • Data Engineer (Python, SQL and AWS) - 7+ Yrs (Relevant - 5 Yrs)

    Work Location - TVM/Kochi/Remote

    NP - Immediate only


    Job Purpose

    We are looking for an experienced Lead Data Engineer with strong expertise in Google Big Query and working knowledge of Looker to support a data modernization engagement. The candidate will lead efforts in revamping data architecture, optimizing data pipelines, and supporting reporting needs, while mentoring an internal client developer to enable smooth transition post-engagement.

    Job Description / Duties and Responsibilities

    Data Architecture & Engineering:

    • Assess and enhance existing data models, pipelines, and architecture in Big Query.

    • Optimize data flows and storage for performance, scalability, and cost-efficiency.

    • Implement best practices in data engineering and governance.

    Data Process & Structure Improvement:

    • Review and improve the current data collection and transformation processes.

    • Ensure high-quality, consistent, and reliable data availability.

    BI Support – Dashboards & Reports:

    • Enhance and maintain existing reports and dashboards using Looker.

    • Design and build new reports as per evolving business requirements.

    Stakeholder Collaboration:

    • Work closely with business and technical stakeholders to gather data and reporting needs.

    • Translate business requirements into technical deliverables.

    Mentoring & Knowledge Transfer:

    • Guide and mentor the client’s internal developer.

    • Ensure sufficient handover for long-term maintainability after project completion.

    Must-Have Skills:

    • Strong experience with Google Big Query – data modeling, query optimization, performance tuning.

    • Proficient in building and managing data pipelines and ETL/ELT workflows.

    • Solid SQL skills and experience working with large datasets.

    • Experience with Looker – creating/modifying dashboards and Look ML understanding.

    • Experience with version control (e.g., Git) and CI/CD for data solutions.

    • Ability to work in Agile environments and with remote teams.

    Good-to-Have Skills:

    • Exposure to GCP services beyond Big Query (e.g., Dataflow, Cloud Functions).

    Soft Skills:

    • Excellent communication and interpersonal skills.

    • Proven ability to work independently in a client-facing role.

    • Experience in mentoring or coaching team members is a plus.

    • To adhere to the Information Security Management policies and procedures.

    This advertiser has chosen not to accept applicants from your region.
     

    Nearby Locations

    Other Jobs Near Me

    Industry

    1. request_quote Accounting
    2. work Administrative
    3. eco Agriculture Forestry
    4. smart_toy AI & Emerging Technologies
    5. school Apprenticeships & Trainee
    6. apartment Architecture
    7. palette Arts & Entertainment
    8. directions_car Automotive
    9. flight_takeoff Aviation
    10. account_balance Banking & Finance
    11. local_florist Beauty & Wellness
    12. restaurant Catering
    13. volunteer_activism Charity & Voluntary
    14. science Chemical Engineering
    15. child_friendly Childcare
    16. foundation Civil Engineering
    17. clean_hands Cleaning & Sanitation
    18. diversity_3 Community & Social Care
    19. construction Construction
    20. brush Creative & Digital
    21. currency_bitcoin Crypto & Blockchain
    22. support_agent Customer Service & Helpdesk
    23. medical_services Dental
    24. medical_services Driving & Transport
    25. medical_services E Commerce & Social Media
    26. school Education & Teaching
    27. electrical_services Electrical Engineering
    28. bolt Energy
    29. local_mall Fmcg
    30. gavel Government & Non Profit
    31. emoji_events Graduate
    32. health_and_safety Healthcare
    33. beach_access Hospitality & Tourism
    34. groups Human Resources
    35. precision_manufacturing Industrial Engineering
    36. security Information Security
    37. handyman Installation & Maintenance
    38. policy Insurance
    39. code IT & Software
    40. gavel Legal
    41. sports_soccer Leisure & Sports
    42. inventory_2 Logistics & Warehousing
    43. supervisor_account Management
    44. supervisor_account Management Consultancy
    45. supervisor_account Manufacturing & Production
    46. campaign Marketing
    47. build Mechanical Engineering
    48. perm_media Media & PR
    49. local_hospital Medical
    50. local_hospital Military & Public Safety
    51. local_hospital Mining
    52. medical_services Nursing
    53. local_gas_station Oil & Gas
    54. biotech Pharmaceutical
    55. checklist_rtl Project Management
    56. shopping_bag Purchasing
    57. home_work Real Estate
    58. person_search Recruitment Consultancy
    59. store Retail
    60. point_of_sale Sales
    61. science Scientific Research & Development
    62. wifi Telecoms
    63. psychology Therapy
    64. pets Veterinary
    View All Lead Data Engineer Jobs