111 Big Data jobs in Chennai

About the Role

We are looking for a highly skilled Senior Software Engineer with strong experience in Flink/Beam, Kafka, and Java/Scala to design, build, and optimize real-time, event-driven streaming systems .

The ideal candidate will have deep knowledge of distributed systems, containerization, and security best practices, preferably in banking or financial environments .

Technology Area

Flink / Beam, Kafka, Java

Education

• Degree or Postgraduate in Computer Science or related field (or equivalent industry experience)

Experience

• Minimum 5+ years of development and design experience in Java/Scala with Flink, Beam (or Spark Streaming) and Kafka .

Technical Skills

• Extensive coding experience and knowledge in Event-driven and Streaming Architecture
• Experience in JVM tuning for performance optimization
• Knowledge of Containerization using Docker and Kubernetes
• Working knowledge on Caching systems , preferably Redis (Nice to have)
• Strong command of Linux OS configuration and Shell scripting
• Good hands-on experience with Design Patterns and their implementation
• Well-versed with CI/CD principles (GitHub, Jenkins, etc.) and troubleshooting in distributed services ecosystems
• Experience working with SQL and NoSQL databases
• Familiar with Distributed services resiliency and monitoring in production environments
• Experience designing, building, testing, and implementing security systems , including identifying design gaps and recommending enhancements
• Adherence to established security and compliance policies , understanding of vulnerabilities and corrective actions
• Knowledge of security controls for data transfer (CRON, ETLs, JDBC-ODBC scripts)
• Understanding of Networking basics including DNS, Proxy, ACL, Policy, and troubleshooting
• Awareness of compliance and regulatory requirements (encryption, anonymization, data integrity, policy control)
• Understanding of data sensitivity in logging and in-memory storage (e.g., ensuring no card numbers or PII in logs)
• Ability to implement wrapper solutions for components lacking security controls to ensure compliance with banking standards

Functional Skills

• Experience working in Agile methodology
• Ensure quality of technical and application architecture and design across systems
• Research and benchmark technologies against best-in-class solutions
• Banking, Financial, or FinTech experience in an enterprise environment is preferred
• Ability to influence multiple teams on technical considerations, sharing deep knowledge and improving team effectiveness
• Self-motivated and proactive , capable of owning tasks independently while collaborating effectively across teams

Hiring!

We are hiring for one of our client!

Position - Big Data Engineer

Experience - 5+years

Location - Chennai, Bangalore

Work Mode - Hybrid

Notice Period - Immediate Joiners(Seving Notice Period within 15days only)

Job Overview

We are seeking a skilled Big Data Developer to join our team in a mid-level position. This full-time , hybrid role is based in Chennai and Bangalore . The ideal candidate will have 4 to 6 years of experience in big data technologies , contributing to developing and optimizing high-quality data pipelines and applications.

Qualifications and Skills

Proficiency in Java (Mandatory skill) is essential for developing robust and scalable data processing applications.

Expertise in Python (Mandatory skill) for efficient data manipulation and automation of data processing tasks.

In-depth knowledge of ETL (Mandatory skill) processes to extract, transform, and load data from various sources into centralized databases.

Experienced in working with Hadoop ecosystems, leveraging distributed computing for processing large datasets efficiently.

Proficient in utilizing Hive for executing queries and managing large datasets in a Hadoop environment.

Skilled in PySpark for real-time data processing and analytics, optimizing data workflows within a big data framework.

Strong command over SQL for database querying, involving complex joins, aggregations, and performance tuning.

Problem-solving and analytical skills to troubleshoot data issues and enhance data architectures.

Roles and Responsibilities

Design and implement scalable data pipelines for batch and stream processing to handle large datasets efficiently.

Collaborate with data scientists and architects to integrate analytics and reporting solutions.

Optimize ETL processes to ensure data accuracy and integrity across various platforms.

Develop and maintain data workflows using Hadoop and related technologies to support data projects.

Enhance data models and data architecture to improve data quality and accessibility.

Identify and resolve data discrepancies and implement enhancements to improve system performance.

Stay updated with the latest big data technologies and incorporate them into current systems when beneficial.

#bigdata #engineer #java #python #hadoop #chennai #bangalore #hybrid #immediate #hiring

Interested candidates share the CV -

Thanks In Advance

Chiselon

The Engineering Analyst 2 is an intermediate level position responsible for a variety of engineering activities including the design, acquisition and development of hardware, software and network infrastructure in coordination with the Technology team. The overall objective of this role is to ensure quality standards are being met within existing and planned frameworks.
**Responsibilities:**
+ Perform system and application monitoring, capacity planning and systems tests to ensure products meet performance requirements
+ Evaluate technologies, develop prototypes, contribute to design issues, and implement solutions
+ Work with various internal and external teams to identify and resolve problems
+ Consult with end users and clients to identify and correct systems problems or propose solutions
+ Assist in the development of software and systems tools used by integration teams to create end user packages
+ Provide support for operating systems and in-house applications, including third party applications, as needed
+ Perform coding, analysis, testing or other appropriate functions in order to identify problems and propose solutions
+ Adhere to Citi technology standards, audit requirements and corporate compliance issues and requirements
+ Apply knowledge of engineering procedures and concepts and basic knowledge of other technical areas to day to day activities
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 2-4 years of relevant experience in an Engineering role
+ Experience working in Financial Services or a large complex and/or global environment
+ Project Management experience
+ Consistently demonstrates clear and concise written and verbal communication
+ Comprehensive knowledge of design metrics, analytics tools, benchmarking activities and related reporting to identify best practices
+ Demonstrated analytic/diagnostic skills
+ Ability to work in a matrix environment and partner with virtual teams
+ Ability to work independently, multi-task, and take ownership of various parts of a project or initiative
+ Ability to work under pressure and manage to tight deadlines or unexpected changes in expectations or requirements
+ Proven track record of operational process change and improvement
+ Deep understanding of retail banking products and functions (Deposits, Savings and Checking, Money market funds, Certificate of Deposits) Payments, Fund Transfers etc)
+ Deep understanding of Card products, associated processes and life cycle.
+ Understanding of Private banking and wealth management
+ Experience in Hadoop technologies, Data warehousing technologies
+ Comfortable in SQL
+ Excellent written and verbal communication skills, with the ability to present complex financial information clearly and concisely to both technical and non-technical audiences.
+ Ability to work effectively both independently and as part of a team. Strong collaboration and relationship-building skills are crucial for success in this role.
+ A self-starter who takes initiative and is driven to achieve results
+ Possesses a strong analytical mindset and meticulous attention to detail to ensure accuracy and completeness in all tasks
Able to adapt quickly to changing priorities and work effectively in a dynamic environment.
+ Snowflake experience
+ Experience in Data lineage identification, DQ rules implementation
+ Risk and Finance Regulatory reports exposure
**Education:**
+ Bachelor's degree/University degree or equivalent experience
---
**Job Family Group:**
Technology
---
**Job Family:**
Systems & Engineering
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.

Role: Big Data Developer

Experience: 6

Location: Chennai/Gurgaon

Required Technical Skill Set:

GCP , HDFS, Unix Script, Autosys ,Hive.

Must Have:

Hands on experience on Spark framework, Unix scripting, Hive queries, writing UDF in Hive. Theortical knowledge and POC alone will not suffice.

Good knowledge in Software Development Life Cycle and Project Development Lifecycle.

Associate should be able to work independently and should have strong debugging skill in both Hive and Spark.

Associate should have experience developing large scale systems, experience debugging and performance tuning, excellent software design skills, communication skills and ability to work with client partners.

At **Citi** we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world's most global banks we're changing how the world does business.
**Shape your Career with Citi**
We're currently looking for a high caliber professional to join our team as ** ** **Officer,** **Data Engineer -C11- Hybrid** based in Chennai, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy decision and manage your financial well-being to help plan for your future.
**The Data Engineer** is accountable for developing high quality data products to support the Bank's regulatory requirements and data driven decision making. A Data Engineer will serve as an example to other team members, work closely with customers, and remove or escalate roadblocks. By applying their knowledge of data architecture standards, data warehousing, data structures, and business intelligence they will contribute to business outcomes on an agile team.
**Responsibilities** & **Qualifications:**
+ 5-8 years of relevant experience in **Big Data** : Experience of 'big data' platforms such as Hadoop, Hive or Snowflake for data storage and processing.
+ Experience in **Big Data, PySpark .**
+ Demonstrated problem-solving and decision-making skills
+ Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
**Education:**
+ Bachelor's degree/University degree or equivalent experience
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.

**Function Overview:**
**Citi's Services Technology** builds and provides strategic technology solutions to support the Services business, which includes integrated Securities Services, Payments and Collections, Liquidity Management, short term financing and Trade Finance solutions to multinational corporations, financial institutions, public sector organizations and increasingly to mid-corporates with global needs.
**Application / Team Overview:**
Within **Citi's Services Technology -** Information management bring a strong a solid data management capability at scale. This includes hybrid platforms to handle huge enterprise datasets - payments, liquidity/cash management, trades, commercial cards, accountings and master reference data, publisher-consumer models, conformance and marts, presentation and visualization layer using dashboards, integration frameworks, strong data governance - catalogue, DQ, Discovery, Entitlement based data access layers and feeding data to Enterprise Data office. Client specific products like CPI, CPOD, VOX and internally technology tools like Bansai, RTI, CIL, REEDS. Also, legacy applications which are up and running for few decades what serves today's clients need from statement, regulatory, trade, liquidity and AML - a true global solutions using ETLs, Data Marts and customized and standard reports.
**Role Description:**
We are seeking a Software/Application Dev Lead Analyst for Global Data Platform team. The successful candidate will interact with development, testing, and production support teams, as well as other departments within Citigroup (such as the System Administrators, Database Administrators, Data Centre Operations, and Change Control groups) for Services Data platforms.
He/she requires exceptional communication skills across both technology and the business and will have a high degree of visibility. The candidate will be a rigorous technical lead with a strong understanding of how to build **scalable, high throughput, enterprise level Billing generation and delivery applications.**
The ideal candidate will be dependable and resourceful software professional who can comfortably work in and lead a large development team in a distributed, dynamic work environment that fosters diversity, teamwork and collaboration. Complying with Citi's System Development Lifecycle and Information Security requirements, he/she will deliver towards all the development and maintenance projects; along with monitoring, updating and communicating project timelines and milestones, obtaining senior management feedback and understanding potential speed bumps and client's true concerns/needs. The ability to work in high-pressured environment is essential.
**Responsibilities:**
+ Manage one or more Applications Development teams to accomplish established goals as well as conduct personnel duties for team (e.g. performance evaluations, hiring and disciplinary actions)
+ Utilize in-depth knowledge and skills across multiple Applications Development areas to provide technical oversight across systems and applications
+ Review and analyze proposed technical solutions for projects
+ Contribute to formulation of strategies for applications development and other functional areas
+ Develop comprehensive knowledge of how areas of business integrate to accomplish business goals
+ Provide evaluative judgment based on analysis of factual data in complicated and unique situations
+ Impact the Applications Development area through monitoring delivery of end results, participate in budget management, and handling day-to-day staff management issues, including resource management and allocation of work within the team/project
+ Ensure essential procedures are followed and contribute to defining standards negotiating with external parties when necessary
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency, as well as effectively supervise the activity of others and create accountability with those who fail to maintain these standards.
**Detail Role Specifications & Requirements**
**Qualifications:**
Bachelor's or master's degree in computer science, Information Technology, or equivalent Minimum 15 years of work experience At least 8 years of experience of leading Enterprise Bigdata Development and Data Science (Analytical) teams Experience developing Applications for High volume instanced payments and/or AML industry is a significant plus Experience working in Banking technology is preferred Very strong Engineer excellence knowledge and experience like Data patterns, Pipelines, DQs, Recons, Governance, Scalable application traits and cloud portable coding Project management, interpersonal, and communication skills, with high attention to details
**Technical / Functional Proficiency:**
**Must-Have** : Experience in building enterprise-grade transaction management and AML system with focus on recovery, stability, reliability, scalability and performance.
**Must-Have** : Strong hand-on seasoned engineer/lead in Bigdata data store application building using enterprise SDLC/Agile methodology, Data modelling/Architect, supporting Operational and production systems
**Must-Have** : Strong technical skills and hand-on in Spark/Scala, Kafka, Hadoop, Couchbase, HBASE and Snowflake. MQs/Kafka/NDM, Autosys, UNIX scripting, data validation, automation
**Must-Have** : Strong Architecture knowledge in design and develop Real-time (Streaming) and batch processing of different data formats/structures. Adhering to strong EE (Engineering excellence) practices
**Must-Have** : Observability specific AI/ML tools to analysis application and business data for continuous improvements of application/tools/frameworks
**Good-To-Have** : Data specific development on Cloud platforms like AWS and GCP
**Good-To-Have** : Experience developing applications for Payment and AML industries
**Good-To-Have** : Experience in Database technologies (Oracle/SQL or Mongo) and development skills (Sql/Stored Proc development including performance tuning)
**Good-To-have** : Experience in working Unix/Linux and Shell scripting
**Good-To-have** : Bigdata HIVE, HDFS, Spark, Kafka, IMPALA, Cloud ecosystems like AWS
**Exposure** : Working on RabbitMQ/ JMS / MQ /Cache DBs/ Hybrid Cloud service integration & applications.
**Leadership Skills:**
**Must-Have** : Demonstrated people and technical management skills, leading a team of 8-10 high caliber developers & software designers
**Must-Have** : Responsible for providing technical leadership of multiple large scale/complex software solutions, even by extending the existing platform through custom development, customization and integration with in-house solutions and vendor solutions leading to a complete platform that caters to key business requirements
**Must-Have:** Experience of developing/mentoring/guide teams consisting of permanent employees and vendors of 15 - 20 developers
**Must-Have:** Stakeholder Management experience to drive discussions between Business, Solutions, Operations and Application Teams
**Must-Have:** Verbal and written communication skills, both technical and functional
**Must-Have:** Fostering culture of Innovation, cutting-edge technologies suite Banking landscape, Passionate to explore new approaches/architecture/solution/automation/refactoring and reengineering.
**Good-To-Have** : Build and sustain strong relationships with the senior business leaders associated with the platform
**Good-To-Have** : Work with onsite development partners to ensure design and coding best practices.
**Good-To-Have** : Work closely with Program Management and Quality Control teams to deliver quality software to agreed project schedules.
**Good-To-Have** : Understanding technology industry trends, how these affect our future solutions and provide strategic direction for the development community.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.

The Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.
**Responsibilities:**
+ Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
+ Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
+ Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
+ Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
+ Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
+ Ensure essential procedures are followed and help define operating standards and processes
+ Serve as advisor or coach to new or lower level analysts
+ Has the ability to operate with a limited level of direct supervision.
+ Can exercise independence of judgement and autonomy.
+ Acts as SME to senior stakeholders and /or other team members.
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 8 plus years of relevant experience
+ Proficiency in Big Data, Spark, PySpark, Scala, Python
+ Familiarity with Data warehouse, data lake, data analytics, cloud computing, snowflake
+ Experience in systems analysis and programming of software applications
+ Experience in managing and implementing successful projects
+ Working knowledge of consulting/project management techniques/methods
+ Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.

At **Citi** we're not just building technology, we're building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world's most global banks we're changing how the world does business.
**Shape your Career with Citi**
We're currently looking for a high caliber professional to join our team as ** ** **Officer, Big** **Data Engineer -C10- Hybrid** based in Chennai, India. Being part of our team means that we'll provide you with the resources to meet your unique needs, empower you to make healthy decision and manage your financial well-being to help plan for your future. For instance:
+ We provide programs and services for your physical and mental well-being including access to telehealth options, health advocates, confidential counseling and more. Coverage varies by country.
**Responsibilities &** **Qualifications:**
+ 2 - 5 years of relevant experience in **Big data** **platforms such as Hadoop, Hive**
+ **Languages** : Proficient in one or more programming languages commonly used in data engineering such as **Python, PySpark.**
+ **Database Management** : **SQL**
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.