8,083 Big Data Technologies jobs in India

We are seeking an experienced Java developer to lead the development of scalable backend services using Spring Boot and optimize database queries.

The ideal candidate will have a strong background in Java 8/17+ and a solid understanding of Big Data technologies such as Hadoop, Apache Spark, Apache Kafka, and Flink.

Job Opportunity:

Data Engineers design, develop and maintain complex data assets. They work closely with stakeholders to understand requirements, create efficient ETL processes and lead data validation and testing.

Key Responsibilities:

• Create scalable ETL processes using big data technologies
• Develop and maintain data integration documentation
• Lead data quality assurance, UAT and regression testing
• Design and optimize data models, including schema development
• Implement automated data pipelines for real-time data flow

Qualifications and Skills:

• Strong expertise in Python and PySpark programming languages
• Able to write Pyspark scripts for developing data workflows
• Proficient in SQL, Hadoop, Hive, Azure, Databricks and Greenplum platforms
• Skilled in querying metadata and tables from various data management systems using SQL
• Familiarity with big data technologies such as Hadoop, Spark and distributed computing frameworks
• Experience with Hue interface and running Hive SQL queries, Apache Oozie job scheduling
• Excellent communication and problem-solving skills
• 4-7 years of experience in Data Engineering field
• Proficiency in common programming languages used in data engineering
• Experience in Azure cloud computing environment

About the Role:

This is an exciting opportunity to join a team of data experts and contribute to the development of innovative data solutions.

Summary

- Bitkraft Technologies LLP is looking for Technical Architect to join our software engineering team
- You will be working across the stack on cutting edge web development projects for our custom services business.
- As a Senior Technical Architect, you play a pivotal role in designing, developing, and implementing cutting-edge data processing solutions
- The ideal candidate will have a deep understanding of distributed systems, big data technologies, and real-time data processing frameworks.
- If you love solving problems, are a team player and want to work in a fast paced environment with core technical and business challenges, we would like to meet you.
- Essential Skills
- Deep understanding of big data technologies, including Hadoop, Spark, Kafka, and Flink.
- Proven experience in designing and implementing scalable, high-performance data processing solutions.
- Strong knowledge of real-time data processing concepts and frameworks.
- Familiarity with GraphQL and its use in API design.
- Familiarity with cloud platforms (AWS, Azure, GCP) and container orchestration (Kubernetes).
- Other Essential Skills / Requirements
- Great attention to detail
- Strong work ethic and commitment to meet deadlines and support team members meet goals
- Be flexible with working across time zones with overseas customers if required
- Ability to work independently and as part of a team.
- Desirable Skills
- Experience with machine learning frameworks and tools.
- Knowledge of data governance and compliance standards.
- Familiarity with CI/CD practices and DevOps methodologies.
- Key Responsibilities
- Provide technical leadership and guidance to development teams, ensuring adherence to best practices and architectural
- standards.
- Design and architect scalable, high-performance data processing solutions using technologies like Kafka, Spark, Stream Processing, batch Processing, Apache Flink, Hadoop, and GraphQL.
- Develop and optimize data pipelines to efficiently extract, transform, and load (ETL) data from various sources into target systems.
- Implement real-time data processing solutions using technologies like Kafka and Flink to enable rapid insights and decision-making.
- Design and implement batch processing workflows using Hadoop or other big data frameworks to handle large datasets.
- Create GraphQL APIs to expose data services, ensuring efficient and flexible data access.
- Research and evaluate new technologies and tools to stay abreast of industry trends and identify opportunities for improvement.
- Monitor and optimize the performance of data processing systems to ensure maximum efficiency and scalability.
- Collaborate with cross-functional teams (e.g., data scientists, data engineers, product managers) to deliver high-quality data solutions.
- Work closely with product management, data science, and operations teams to understand requirements and deliver data solutions that align with business goals.
- Experience 5 to 10 years
- About Bitkraft Technologies LLP
- Bitkraft Technologies LLP is an award winning Software Engineering Consultancy focused on Enterprise Software Solutions, Mobile Apps Development, ML/AI Solution Engineering, Extended Reality, Managed Cloud Services and Technology Skill-sourcing, with an extraordinary track record.
- We are driven by technology and push the limits of what can be done to realise the business needs of our customers.
- Our team is committed towards delivering products of the highest standards and we take pride in creating robust user-driven solutions that meet business needs.
- Bitkraft boasts of clients across over 10+ countries including US, UK, UAE, Oman, Australia and India to name a few.
- )

Job Description:

• Deliver scalable backend services using Java and Spring Boot.
• Design and implement RESTful APIs to manage request/response lifecycle.
• Apply efficient coding practices with Java features (Streams, Lambdas, Optional).
• Work with SQL (Oracle) and NoSQL databases (e.g., MongoDB, Cassandra) for data storage and retrieval.
• Optimize database queries using Oracle SQL Hints and performance techniques.
• Collaborate with cross-functional teams to deliver high-quality software solutions.
• Participate in code reviews and technical discussions.
• Explore and integrate event-based frameworks for asynchronous processing.
• Contribute to the design and implementation of Big Data pipelines.

Required Skills:

• 6-9 years of experience in Big Data technologies.

Benefits:

• Opportunity to work on impactful projects.
• Chance to grow as a professional and individual.
• Collaborative work environment.

About Us:

• Looking for candidates with experience in Big Data technologies.

LTIMindtree hiring for Celonis- Mumbai/Chennai location.

Exp-8 to 12 yrs

Notice period-immediate to 30 days

Location- Mumbai/Chennai

Mandatory skills-

• Celonis process mining tool, SQL, PQL ETL, Basic knowledge of Data Engineering, analytics
• Nice to have Technical Skills: Experience of Celonis with SAP

If interested Share me these details along with cv in my email -

Total Experience in Celonis-

Relevant SQL and ETL -

Current CTC-

Expected CTC-

Holding offers if any-

Current Location-

Preferred Location-

Notice period-

Skills-

Current Company-

Date of Birth-

Pan no-

Share your passport size photo-

Availability for interview -

Job Description-

Knowledge with SQL PQL ETL

Experience on Celonis Process mining tool

Basic knowledge of Data engineering analytics

Should have exposure to International Clients

Basic understanding of Business Processes

Responsibilities:

• Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.
• Create the data integration and data diagram documentation.
• Lead the data validation, UAT and regression test for new data asset creation.
• Create and maintain data models, including schema design and optimization.
• Create and manage data pipelines that automate the flow of data, ensuring data quality and consistency.

Qualifications and Skills:

• Strong knowledge on Python and Pyspark
• Expectation is to have ability to write Pyspark scripts for developing data workflows.
• Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum
• Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum.
• Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks.
• Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows.
• Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.
• Expectation is to have strong problem-solving and troubleshooting skills.
• Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes.
• Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
• 3-7 years of experience in Data Engineer.
• Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL.
• Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks.
• Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

Are you ready to make an impact? ZF is looking for talented individuals to join our team. As a FutureStarter, you'll have the opportunity to shape the future of mobility. Join us and be part of something extraordinary

Country/Region: IN

Location:
Pune, MH, IN,

Req ID 81572 | Pune, India, ZF India Pvt. Ltd.

About the team

ZF Aftermarket is a EUR 3 billion powerhouse with a combined team of about 8,000 employees. We have a total of 120 locations in 40 countries worldwide – including 90 logistics centers – and more than 650 service partners with a strong presence in both automotive and industrial markets. This makes us the second largest aftermarket organization worldwide.

What you can look forward to as Data Engineer & Analytics Specialist

• KPI Reporting & Business Alignment: Develop and maintain KPI reports in close collaboration with Senior Management and SME, ensuring they drive strategic decision-making supported by clear storytelling and actionable dashboards
• Business Needs Translation: Investigate and translate business requirements, ensuring that technical outputs align with business goals and are usable by stakeholders
• Stakeholder Collaboration: Build strong relationships with business teams to understand their challenges and bridge the gap between business and technical teams
• Data Visualization & Communication: Create interactive dashboards and reports using Power BI, ensuring they are accessible and valuable to both subject matter experts and business users
• Vision for Business Outputs: Define the end-user experience for data products, ensuring clarity, usability, and strategic alignment
• Data Transformation & Integration: Translate business requirements into data requirement, process, clean, and integrate data from various sources to meet business requirements using Python, SQL and Power BI

Your profile as Data Engineer & Analytics Specialist

• Strong domain knowledge in B2B portfolio management
• Total experience of 6-10 years in Data engineering/ Advanced analytics roles.
• Proven experience in building / setting up KPI reporting with Power BI & SQL (Python / pyspark highly valued) supported by strong visualizations
• Significant experience setting up customer engagement platforms and driving digital transformation initiatives. Ability to manage programs and targets with rigor and strategic oversight
• Strong leadership, communication, and problem-solving skills. Experience working in a remote, matrix-based organization and leading distributed project teams
• Entrepreneurial mindset with resilience to overcome roadblocks and drive change

Why you should choose ZF in India

• Innovative Environment: ZF is at the forefront of technological advancements, offering a dynamic and innovative work environment that encourages creativity and growth.
• Diverse and Inclusive Culture: ZF fosters a diverse and inclusive workplace where all employees are valued and respected, promoting a culture of collaboration and mutual support.
• Career Development: ZF is committed to the professional growth of its employees, offering extensive training programs, career development opportunities, and a clear path for advancement.
• Global Presence: As a part of a global leader in driveline and chassis technology, ZF provides opportunities to work on international projects and collaborate with teams worldwide.
• Sustainability Focus: ZF is dedicated to sustainability and environmental responsibility, actively working towards creating eco-friendly solutions and reducing its carbon footprint.
• Employee Well-being: ZF prioritizes the well-being of its employees, providing comprehensive health and wellness programs, flexible work arrangements, and a supportive work-life balance.

Be part of our ZF team as Data Engineering & Analytical Consultant and apply now

Contact

Veerabrahmam Darukumalli

At ZF, we continuously strive to build and maintain a culture where inclusiveness is lived and diversity is valued. We actively seek ways to remove barriers so that all our employees can rise to their full potential. We aim to embed this vision in our legacy through how we operate and build our products as we shape the future of mobility.

Find out how we work at ZF:

Job Segment: R&D Engineer, Analytics, Database, SQL, User Experience, Engineering, Management, Technology

Strong Skill in Palantir, Strong SQL, Python, and data engineering background

Hands-on experience in Foundry data pipelines and ontology

Good knowledge of data visualization and operational workflows

Awareness of cloud, governance, and DevOps concepts Data modeling, schema design, data lineage, metadata management APIs, RESTful services, data connectors

Role & responsibilities

Experienced Data Warehouse Developer/DBA with expertise in building data pipelines, curating data, modeling data for analytics, and providing subject matter expertise with experience implementing end-to-end data warehouse/data lake solutions including ETL development, data profiling, data curation, data modeling and deployment with expertice in Postgres, Reddit, Redis, C#, Python.

• Expert in Databases creating, governance and ETL development.
• Expert in Postgres, Reddit, Redis, C#, Python
• Proven track record of data warehouse/data lake data development including building data pipelines and data modeling.
• Strong experience in SQL, data curation, ETL tooling and pipeline development.
• Hands-on experience with RDBMS/DWs and writing ETL pipelines.
• Technical skills and experience using relational databases (e.g. Oracle InForm, Oracle DMW, MS SQL Server or MS Access)
• Domain knowledge in the pharmaceutical industry and GxP preferred.