24 Apache Nifi jobs in India

Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at

Job Title: Apache Nifi Kafka Engineer

Experience: 5- 8 Years

Category: Software Development/ Engineering

Shift: General

Main location: India, Tamil Nadu, Chennai

Position ID: J

Employment Type: Full Time

Education Qualification: Bachelor's degree in computer science or related field or higher with minimum 5 years of relevant experience.

Position Description: We are seeking a highly experienced Senior Data Engineer with a strong background in building end-to-end ETL and ELT data pipelines and working across both AWS and GCP cloud platforms. The ideal candidate will have deep expertise in Big Data ecosystems, real-time and batch processing, and experience working with large-scale distributed systems.

Your future duties and responsibilities

• Design, develop, and deploy end-to-end scalable data pipelines for ingesting, transforming, and storing structured, semi-structured, and unstructured data.
• Work with large-scale data processing frameworks like PySpark, Spark SQL, and Spark Streaming to enable real-time and batch analytics.
• Optimize and manage ETL pipelines, data lakes and data warehouses on AWS (S3, Athena, Redshift, RDS, Glue, EMR) and GCP (BigQuery, DataProc, GCS).
• Build and optimize ETL/ELT workflows for ingesting, transforming, and delivering high-volume data from diverse sources to data lakes and data warehouses
• Develop data integration workflows using tools like Apache NiFi, Kafka, Sqoop, Hadoop, HDFS, Hive, Airflow.
• Implement CDC, SCD Type 1, SCD Type 2, and real-time ingestion patterns.
• Leverage Hive for data querying and performance tuning using Partitioning, Bucketing, and Compression techniques.
• Handle multiple data formats including Parquet, ORC, Avro, JSON, XML, and CSV.
• Collaborate with Data Scientists, Analysts, and Product teams to translate business requirements into technical solutions.
• Mentor junior engineers, conduct technical interviews, and contribute to building a high-performance data engineering team.
• Exposure to CI/CD pipelines and container orchestration (Docker, Kubernetes).
• Participate in Agile discussions, client meetings, sprint planning, and demos.

Required qualifications to be successful in this role

Must-Have Skills:

• Proficient in PySpark, Spark Core, Spark SQL, and Spark Streaming.
• Hands-on experience with AWS data services – S3, Redshift, Athena, Glue, EMR.
• Strong exposure to Google Cloud Platform – BigQuery, DataProc, Cloud Spanner, GCS.
• Expertise in Hive, HDFS, Kafka, Apache NiFi and Sqoop.
• Experience with job orchestration tools like Apache Airflow or BMC Control-M.
• Solid understanding of data modeling, data governance, and data quality best practices.
• Strong problem-solving and communication skills with a collaborative mindset.

CGI is an equal opportunity employer. In addition, CGI is committed to providing accommodation for people with disabilities in accordance with provincial legislation. Please let us know if you require reasonable accommodation due to a disability during any aspect of the recruitment process and we will work with you to address your needs.

Together, as owners, let's turn meaningful insights into action.

Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because…

You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction.

Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise.

You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

Come join our team—one of the largest IT and business consulting services firms in the world.

Position Overview

We are seeking a highly skilled 
Data Engineer
 with strong expertise in 
Apache NiFi
 and 
Groovy scripting
 to design and implement robust data pipelines. The role involves building scalable data flows, automating ingestion and transformation, and integrating multiple enterprise systems using NiFi and Groovy.

Key Responsibilities

• Design, develop, and maintain 
  end-to-end data pipelines
   using Apache NiFi.
• Write and maintain 
  Groovy scripts
   for advanced data transformation, parsing, validation, and enrichment.
• Build reusable components and templates for 
  data ingestion, routing, and processing
  .
• Integrate NiFi with 
  databases, APIs, messaging systems (Kafka, JMS, MQTT), and cloud platforms (AWS/Azure/GCP)
  .
• Implement 
  data quality checks, schema validation, and error handling
   within NiFi flows.
• Ensure 
  data lineage, provenance, and governance compliance
   across all pipelines.
• Monitor, troubleshoot, and optimize NiFi flows for 
  scalability, performance, and reliability
  .
• Work closely with Data Scientists, Analysts, and Architects to deliver business-ready datasets.
• Support 
  CI/CD automation for NiFi pipelines
   and version control using Git.

Required Qualifications

• Bachelor's degree in Computer Science, Data Engineering, or related field.
• 3+ years of hands-on experience
   in building pipelines with 
  Apache NiFi
  .
• Strong expertise in 
  Groovy scripting
   for NiFi data transformations and custom logic.
• Experience with 
  data formats
  : JSON, XML, CSV, Avro, Parquet.
• Solid understanding of 
  databases (SQL/NoSQL)
   and API integrations.
• Familiarity with 
  Linux/Unix environments
  , shell scripting, and automation.
• Experience with 
  version control (Git/GitLab/Bitbucket)
  .

Preferred / Nice to Have

• Exposure to 
  real-time streaming platforms
   (Kafka, Spark Streaming, Flink).
• Experience with 
  cloud data services
   (AWS Glue, S3, Kinesis, Azure Data Factory, GCP Dataflow).
• Knowledge of 
  DevOps practices
   (Docker, Kubernetes, Jenkins, Ansible).
• Familiarity with 
  monitoring/observability tools
   (Prometheus, Grafana, ELK stack).
• Awareness of 
  data governance, security, and compliance standards
   (GDPR, HIPAA).

Soft Skills

• Strong analytical and problem-solving ability.
• Ability to translate business requirements into 
  data pipelines and flows
  .
• Collaborative mindset for working with cross-functional teams.
• Self-motivated and eager to learn emerging data engineering tools.

Compensation & Benefits

• Competitive salary and benefits package.
• Health insurance and employee wellness programs.
• Hybrid/remote working options.
• Professional development and certification sponsorship.
• Collaborative and innovative work culture.

Freelance Interviewer-Apache Nifi

Experience:7 to 10 years

About the company

We are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.

Were looking for experienced professionals across various domains who can take up freelance interviews for our clients. Interviews are conducted remotely, and schedules are flexible based on your availability. And currently we are looking at panels for the given description.

Experience

• Minimum 5 years experience in implementing end to end integration solutions using NiFi processors
• Minimum 5 years experience in Java and Spring boot with Microservices
• Minimum 3 years experience in application security such as SSL Certificates, cryptography
• Minimum 2 years experience in Distributed architecture

Technical Skills

• Excellent in designing and developing NiFi and MiNiFi Flows using various processors along with failover scenarios
• Excellent in SSL Certificates, communicate protocols such as SFTP, Site to Site and cryptography
• Well versed in distributed architecture using zookeeper
• Excellent in Java and microservices
• Familiar with Distributed services resiliency and monitoring in a production environment.

Functional Skills

• Experience in following best Coding, Security, Unit testing and Documentation standards and practices
• Experience in Banking/Financial domain is highly desired
• Experience in Agile methodology
• Ensure quality of technical and application architecture and design of systems across the organization
• Effectively research and benchmark technology against other best in class technologies

Soft Skills

• Should be excellent in communication, should have positive attitude towards work and should be eager to learn new things
• Self-motivator and self-starter, Ability to own and drive things without supervision and works collaboratively with the teams across the organization
• Should have excellent interpersonal skills to interact and present the ideas to senior management in IT and Business alike
• Should be able to train/mentor team members

Freelance Interviewer-Apache Nifi

Experience:7 to 10 years

About the company

We are an HR Tech company based in Trivandrum, offering hiring support to MNCs across India through interview ,assessments and recruitment services. We have a network of 4000+ experienced professionals who take interviews in their available time slots.

Were looking for experienced professionals across various domains who can take up freelance interviews for our clients. Interviews are conducted remotely, and schedules are flexible based on your availability. And currently we are looking at panels for the given description.

Experience

• Minimum 5 years experience in implementing end to end integration solutions using NiFi processors
• Minimum 5 years experience in Java and Spring boot with Microservices
• Minimum 3 years experience in application security such as SSL Certificates, cryptography
• Minimum 2 years experience in Distributed architecture

Technical Skills

• Excellent in designing and developing NiFi and MiNiFi Flows using various processors along with failover scenarios
• Excellent in SSL Certificates, communicate protocols such as SFTP, Site to Site and cryptography
• Well versed in distributed architecture using zookeeper
• Excellent in Java and microservices
• Familiar with Distributed services resiliency and monitoring in a production environment.

Functional Skills

• Experience in following best Coding, Security, Unit testing and Documentation standards and practices
• Experience in Banking/Financial domain is highly desired
• Experience in Agile methodology
• Ensure quality of technical and application architecture and design of systems across the organization
• Effectively research and benchmark technology against other best in class technologies

Soft Skills

• Should be excellent in communication, should have positive attitude towards work and should be eager to learn new things
• Self-motivator and self-starter, Ability to own and drive things without supervision and works collaboratively with the teams across the organization
• Should have excellent interpersonal skills to interact and present the ideas to senior management in IT and Business alike
• Should be able to train/mentor team members

Job description

We're Hiring: NiFi Data Engineer (Contract | 6–10 Years Experience)

Location: Hyderabad (Preferred) | Bangalore | Mumbai | Chennai | Ahmedabad | Pune

Work Mode: Hybrid – 2 Days/Week from Mindtrilogy Office (Mandatory)

Notice Period: Immediate Joiners Only

No. of Positions: 2

Job Type: Contract

About the Role

Mindtrilogy is seeking experienced NiFi Data Engineers (Contract) to join our data engineering team. In this role, you'll be responsible for designing and maintaining scalable data pipelines using Apache NiFi, enabling efficient data processing and transformation across our systems.

You'll work closely with cross-functional teams to power innovative solutions that drive real business outcomes.

Key Responsibilities Core Responsibilities

Design and build robust, high-performance data pipelines using Apache NiFi

Write and maintain transformation scripts in Java and Python

Develop and manage ETL/ELT workflows for diverse data sources

Ensure reliable data ingestion, transformation, and distribution processes

Collaborate with Engineers, Developers, and Product Managers to align on data requirements

Cloud & Infrastructure

Utilize AWS services including S3, Glue, Athena, EMR, and AWS Batch

Apply data security best practices (e.g., SSL/TLS, client authentication, validation, logging)

Monitoring & Optimization

Implement monitoring, logging, and alerting for production pipelines

Troubleshoot and resolve performance and reliability issues proactively

Required Skills & Qualifications

Strong expertise in Apache NiFi and building scalable data pipelines

Proficient in Java, Python, and SQL

Experience with AWS data stack (S3, Glue, EMR, Athena, Batch)

Solid scripting skills using Groovy or Python for custom data transformations

Strong grasp of data modeling, API design, and data governance

Familiarity with implementing security protocols and best practices

Soft Skills & Culture Fit

Strong communication and team collaboration skills

Self-driven with a proactive mindset

Ability to thrive in a hybrid work environment

Flexible, adaptable, and eager to take ownership

Apache NiFi, ETL/ELT processes, MySQL, PostgreSQL,AWS, Azure, GCP, Python, Bash, Hadoop, Spark, Docker, Kubernetes, GDPR, HIPAA

About Our Client

A digital services company, part of a large Indian conglomerate, focuses on creating digital products and services for consumers and businesses

Job Description

The Successful Applicant

Job Summary:

The Data Flow Engineer's primary role is to create and manage data connections, perform validations, and execute transformations. Their work is integral to the ongoing process of iterative improvement, with a particular focus on enhancing auto-reconciliation within the system through advanced technology.

Responsibilities:

• Import and validate file delivery for new clients.
• Automate daily process monitoring and reporting.
• Establish connections through external APIs and FTPs.
• Ensure timely and dependable consumption of external portfolio data.
• Normalize external datasets into a standardized Clearwater format facilitating the in-take process.
• Mine data from existing feeds to identify, design, and implement solutions to improve auto-reconciliation.
• Execute improvements requested from Operations and Development groups.
• Apply acquired skills, procedures, and decision-making best practices to complete various issues, such as normalizing new feeds and improving automation.
• Understand and reference or explain the general workflow, tools, and Clearwater value proposition.
• Use critical thinking to address issues and offer solutions for both internal and external parties, ensuring best practices are employed.
• Clearly and effectively communicate the technical aspects of Clearwater systems and our best practices with non-technical internal and external stakeholders.
• Engage in light on-call duties.

Required Skills:

• Securities, accounting, and financial experience.
• Strong understanding of SQL and relational database principles.
• Experience with scripting programming languages like Groovy, Perl or Python.
• Experience with industry-standard data transmission protocols preferred.
• Securities, accounting, and financial experience preferred.
• Strong computer skills, including proficiency in Microsoft Office.
• Excellent attention to detail and strong documentation skills.
• Outstanding verbal and written communication skills.
• Strong organizational and interpersonal skills.
• Exceptional problem-solving abilities.

Education and Experience:

• Bachelor's degree in Math, Computer Information Systems, or other relevant degrees.
• 2+ years of relevant experience.
• Experience with industry-standard data transmission protocols.