3,016 Data Pipelines jobs in India

Team: Development - Alpha Data

Position Overview:

We are seeking an experienced Python developer to join our Alpha Data team, responsible for delivering a vast quantity of data served to users worldwide. You will be a cornerstone of a growing Data team, becoming a technical subject matter expert and developing strong working relationships with quant researchers, traders, and fellow colleagues across our Technology organisation.

Alpha Data teams are able to deploy valuable data to the rest of the Squarepoint business at speed. Ingestion pipelines and data transformation jobs are resilient and highly maintainable, while the data models are carefully designed in close collaboration with our researchers for efficient query construction and alpha generation.

We achieve an economy of scale through building new frameworks, libraries, and services used to increase the team's quality of life, throughput, and code quality. Teamwork and collaboration are encouraged, excellence is rewarded and diversity of thought and creative solutions are valued. Our emphasis is on a culture of learning, development, and growth.

Required Qualifications:

Nice to haves

HEROIC Cybersecurity ( HEROIC ) is seeking a Senior Data Infrastructure Engineer with deep expertise in DataStax Enterprise (DSE) and Apache Cassandra to help architect, scale, and maintain the data infrastructure that powers our cybersecurity intelligence platforms.

You will be responsible for designing and managing fully automated, big data pipelines that ingest, process, and serve hundreds of billions of breached and leaked records sourced from the surface, deep, and dark web. You'll work with DSE Cassandra, Solr, and Spark, helping us move toward a 99% automated pipeline for data ingestion, enrichment, deduplication, and indexing — all built for scale, speed, and reliability.

This position is critical in ensuring our systems are fast, reliable, and resilient as we ingest thousands of unique datasets daily from global threat intelligence sources.

What you will do: 

• Design, deploy, and maintain high-performance Cassandra clusters using DataStax Enterprise (DSE)
  
• Architect and optimize automated data pipelines to ingest, clean, enrich, and store billions of records daily
  
• Configure and manage DSE Solr and Spark to support search and distributed processing at scale
  
• Automate dataset ingestion workflows from unstructured surface, deep, and dark web sources
  
• Cluster management, replication strategy, capacity planning, and performance tuning
  
• Ensure data integrity, availability, and security across all distributed systems
  
• Write and manage ETL processes, scripts, and APIs to support data flow automation
  
• Monitor systems for bottlenecks, optimize queries and indexes, and resolve production issues
  
• Research and integrate third-party data tools or AI-based enhancements (e.g., smart data parsing, deduplication, ML-based classification)
  
• Collaborate with engineering, data science, and product teams to support HEROIC’s AI-powered cybersecurity platform
  

• Minimum 5 years experience with Cassandra / DataStax Enterprise in production environments
  
• Hands-on experience with DSE Cassandra, Solr, Apache Spark, CQL, and data modeling at scale
  
• Strong understanding of NoSQL architecture, sharding, replication, and high availability
  
• Advanced knowledge of Linux/Unix, shell scripting, and automation tools (e.g., Ansible, Terraform)
  
• Proficient in at least one programming language: Python, Java, or Scala
  
• Experience building large-scale automated data ingestion systems or ETL workflows
  
• Solid grasp of AI-enhanced data processing, including smart cleaning, deduplication, and classification
  
• Excellent written and spoken English communication skills
  
• Prior experience with cybersecurity or dark web data (preferred but not required)
  

• Position Type: Full-time
  
• Location: Pune, India  (Remote – Work from anywhere)
  
• Compensation: Competitive salary depending on experience
  
• Benefits: Paid Time Off + Public Holidays
  
• Professional Growth: Amazing upward mobility in a rapidly expanding company.
  

• Innovative Culture: Fast-paced, innovative, and mission-driven. Be part of a team that leverages AI and cutting-edge technologies. 
  

   
  

About Us: HEROIC Cybersecurity ( HEROIC ) is building the future of cybersecurity. Unlike traditional cybersecurity solutions, HEROIC takes a predictive and proactive approach to intelligently secure our users before an attack or threat occurs. Our work environment is fast-paced, challenging and exciting. At HEROIC, you’ll work with a team of passionate, engaged individuals dedicated to intelligently securing the technology of people all over the world.

Position Keywords: DataStax Enterprise (DSE), Apache Cassandra, Apache Spark, Apache Solr, AWS, Jira, NoSQL, CQL (Cassandra Query Language), Data Modeling, Data Replication, ETL Pipelines, Data Deduplication, Data Lake, Linux/Unix Administration, Bash, Docker, Kubernetes, CI/CD, Python, Java, Distributed Systems, Cluster Management, Performance Tuning, High Availability, Disaster Recovery, AI-based Automation, Artificial Intelligence, Big Data, Dark Web Data

Job Description

We’re AtkinsRéalis, a world-leading Design, Engineering and Project Management organization. Created by the integration of long-standing organizations dating back to 1911, we are a world-leading professional services and project management company dedicated to engineering a better future for our planet and its people. We create sustainable solutions that connect people, data and technology to transform the world's infrastructure and energy systems. We deploy global capabilities locally to our clients and deliver unique end-to-end services across the whole life cycle of an asset including consulting, advisory & environmental services, intelligent networks & cybersecurity, design & engineering, procurement, project & construction management, operations & maintenance, decommissioning and capital. The breadth and depth of our capabilities are delivered to clients in key strategic sectors such as Engineering Services, Nuclear, Operations & Maintenance and Capital.

News and information are available at or follow us on LinkedIn.

Our teams are proud to deliver on some of the most prestigious projects across the world. It's thanks to our talented people and their diverse thinking, expertise, and knowledge. Join us and you'll be part of our genuinely collaborative environment, where everyone is supported to make the most of their talents and expertise.
When it comes to work-life balance, AtkinsRéalis is a great place to be. So, let's discuss how our flexible and remote working policies can support your priorities. We're passionate about are work while valuing each other equally. So, ask us about some of our recent pledges for Women's Equality and being a 'Disability Confidence' and 'Inclusive Employer’.

EAI-AtkinsRéalis is a vibrant and continuously growing team. It is an important part of GTC-AtkinsRéalis and widely recognized for its high and quality project deliveries. This would be a vital role to take EAI one step forward in providing data solutions to our business and client. This role would simultaneously work on multiple projects and would provide planning, designing and delivery of data driven projects. Effective communication and a team player are important characteristics of this role.

Key Activities for This Role

Experience & Skills Required:

What We Can Offer You

Why work for AtkinsRéalis?

We at AtkinsRéalis are committed to developing its people both personally and professionally. Our colleagues have the advantage of access to a high ranging training portfolio and development activities designed to help make the best of individual’s abilities and talents. We also actively support staff in achieving corporate membership of relevant institutions.

Meeting Your Needs

To help you get the most out of life in and outside of work, we offer employees ‘Total Reward’.
Making sure you're supported is important to us. So, if you identify as having a disability, tell us ahead of your interview, and we’ll discuss any adjustments you might need.
Additional Information
We are an equal opportunity, drug-free employer committed to promoting a diverse and inclusive community - a place where we can all be ourselves, thrive and develop. To help embed inclusion for all, from day one, we offer a range of family friendly, inclusive employment policies, flexible working arrangements and employee networks to support staff from different backgrounds. As an Equal Opportunities Employer, we value applications from all backgrounds, cultures and ability.

We care about your privacy and are committed to protecting your privacy. Please consult our Privacy Notice on our Careers site to know more about how we collect, use and transfer your Personal Data.

Link: Equality, diversity & inclusion | Atkins India (atkinsrealis.com)

Worker Type

Job Type

At AtkinsRéalis, we seek to hire individuals with diverse characteristics, backgrounds and perspectives. We strongly believe that world-class talent makes no distinctions based on gender, ethnic or national origin, sexual identity and orientation, age, religion or disability, but enriches itself through these differences.

Explore your next opportunity at a Fortune Global 500 organization. Envision innovative possibilities, experience our rewarding culture, and work with talented teams that help you become better every day. We know what it takes to lead UPS into tomorrow—people with a unique combination of skill + passion. If you have the qualities and drive to lead yourself or teams, there are roles ready to cultivate your skills and take you to the next level.

Job Description:

Job Summary
UPS Marketing team is looking for a talented and driven Data Scientist to drive its strategic objectives in the areas of pricing, revenue management, market analysis and evidence/data-based decision making. This role will work across multiple channels and teams to drive tangible results in the organization. You will focus on developing metrics for multiple channels and markets, applying advanced statistical modeling where appropriate and pioneering new analytical methods in a variety of fast paced and rapidly evolving consumer channels. This high visibility position will work with multiple levels of the organization, including senior leadership to bring analytical capabilities to the forefront of pricing, rate setting, and optimization of our go-to-market offers. You will contribute to rapidly evolving UPS Marketing analytical capabilities by working amongst a collaborative team of Data Scientists, Analysts and multiple business stakeholders.

Responsibilities:

• Become a subject matter expert on UPS business processes, data and analytical capabilities to help define and solve business needs using data and advanced statistical methods

• Analyze and extract insights from large-scale structured and unstructured data utilizing multiple platforms and tools.

• Understand and apply appropriate methods for cleaning and transforming data

• Work across multiple stake holders to develop, maintain and improve models in production

• Take the initiative to create and execute analyses in a proactive manner

• Deliver complex analytical and visualizations to broader audiences including upper management and executives

• Deliver analytics and insights to support strategic decision making

• Understand the application of AI/ML when appropriate to solve complex business problems

Qualifications

• Expertise in R, SQL, Python.

• Strong analytical skills and attention to detail. 

• Able to engage key business and executive-level stakeholders to translate business problems to high level analytics solution approach.

• Expertise with statistical techniques, machine learning or operations research and their application in business applications.

• Deep understanding of data management pipelines and experience in launching moderate scale advanced analytics projects in production at scale.

• Proficient in Azure, Google Cloud environment

• Experience implementing open-source technologies and cloud services; with or without the use of enterprise data science platforms.

• Solid oral and written communication skills, especially around analytical concepts and methods. 

• Ability to communicate data through a story framework to convey data-driven results to technical and non-technical audience.

• Master’s Degree in a quantitative field of mathematics, computer science, physics, economics, engineering, statistics (operations research, quantitative social science, etc.), international equivalent, or equivalent job experience.

Bonus Qualifications

• Experience with pricing methodologies and revenue management

• Experience using PySpark, Azure Databricks, Google BigQuery and Vertex AI

• Creating and implementing NLP/LLM projects

• Experience utilizing and applying neurals networks and other AI methodologies

• Familiarity with Data architecture and engineering

Employee Type:
 

UPS is committed to providing a workplace free of discrimination, harassment, and retaliation.

Calling all innovators – find your future at Fiserv.

We're Fiserv, a global leader in Fintech and payments, and we move money and information in a way that moves the world. We connect financial institutions, corporations, merchants, and consumers to one another millions of times a day – quickly, reliably, and securely. Any time you swipe your credit card, pay through a mobile app, or withdraw money from the bank, we're involved. If you want to make an impact on a global scale, come make a difference at Fiserv.

Job Title

Specialist, Data Architecture

• Experience on SSIS/SQL(4-7) years and will be responsible for the development of ETL and Reporting solutions
• Strong Knowledge of SSIS packages, design principles & best practices.
• Experience with requirements gathering, technical analysis, and writing technical specifications
• Candidate must have strong database fundamentals.
• Must have good knowledge of Data Warehousing & Data Modelling Concepts.
• Good communication skills are required.
• Capability to work in a distributed team environment with minimal supervision is required for this profile.
• The position doesn't require working in shifts, however flexibility to overlap with US hours is required.
• Should have good knowledge in writing SQL commands, queries and stored procedures
• Good Knowledge of Snowflake would be preferred.
• Good Knowledge of Python/Pyspark would be preferred

Thank you for considering employment with Fiserv. Please:

• Apply using your legal name
• Complete the step-by-step profile and attach your resume (either is acceptable, both are preferable).

Our commitment to Diversity and Inclusion:

Fiserv is proud to be an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, gender, gender identity, sexual orientation, age, disability, protected veteran status, or any other category protected by law.

Note to agencies:

Fiserv does not accept resume submissions from agencies outside of existing agreements. Please do not send resumes to Fiserv associates. Fiserv is not responsible for any fees associated with unsolicited resume submissions.

Warning about fake job posts:

Please be aware of fraudulent job postings that are not affiliated with Fiserv. Fraudulent job postings may be used by cyber criminals to target your personally identifiable information and/or to steal money or financial information. Any communications from a Fiserv representative will come from a legitimate Fiserv email address.

Calling all innovators – find your future at Fiserv.
We're Fiserv, a global leader in Fintech and payments, and we move money and information in a way that moves the world. We connect financial institutions, corporations, merchants, and consumers to one another millions of times a day – quickly, reliably, and securely. Any time you swipe your credit card, pay through a mobile app, or withdraw money from the bank, we're involved. If you want to make an impact on a global scale, come make a difference at Fiserv.

Job Title
Specialist, Data Architecture

What does a great AI/ML Engineer do?
We are seeking a highly skilled Generative AI Specialist with 6-8 years of extensive experience in AI and machine learning, focusing on generative models and prompt engineering. The successful candidate will work collaboratively within our team to design, implement, and optimize generative AI solutions that enhance our product offerings and provide value to our clients.

What You Will Do

• Develop, implement, and optimize generative models (e.g., GANs, VAEs) for various applications.
• Conduct research and stay updated with the latest advancements in generative AI and deep learning.
• Collaborate with cross-functional teams to identify opportunities for leveraging generative AI in our products.
• Design and optimize prompts used in AI models to improve output quality and relevance.
• Analyze model performance and user data to refine prompt strategies.
• Analyze and preprocess large datasets to train and validate models effectively.
• Create and maintain documentation for algorithms, methodologies, and processes.
• Provide training and support to internal teams on generative AI technologies.

What You Will Need To Have

• Bachelor's or Master's degree in Computer Science, Data Science, AI, or a related field.
• 6-8 years of experience in AI/ML, with a strong focus on generative models and prompt engineering.
• Proficiency in programming languages such as Python or R.
• Experience with deep learning frameworks (e.g., TensorFlow, PyTorch) and libraries.
• Strong understanding of machine learning algorithms and statistical analysis.
• Excellent problem-solving skills and ability to work in a fast-paced environment.
• Strong communication skills to articulate complex ideas to non-technical stakeholders.

What Would Be Great To Have

• Experience in the financial services industry is a plus.
• Familiarity with natural language processing (NLP) or computer vision (CV) applications.
• Contributions to open source projects or published research in generative AI.

Thank You For Considering Employment With Fiserv. Please

• Apply using your legal name
• Complete the step-by-step profile and attach your resume (either is acceptable, both are preferable).

Our Commitment To Diversity And Inclusion
Fiserv is proud to be an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, gender, gender identity, sexual orientation, age, disability, protected veteran status, or any other category protected by law.

Note To Agencies
Fiserv does not accept resume submissions from agencies outside of existing agreements. Please do not send resumes to Fiserv associates. Fiserv is not responsible for any fees associated with unsolicited resume submissions.

Warning About Fake Job Posts
Please be aware of fraudulent job postings that are not affiliated with Fiserv. Fraudulent job postings may be used by cyber criminals to target your personally identifiable information and/or to steal money or financial information. Any communications from a Fiserv representative will come from a legitimate Fiserv email address.