11,591 Big Data jobs in India
Big Data Engineer
Posted today
Job Viewed
Job Description
**Responsibilities:**
+ Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements
+ Identify and analyze issues, make recommendations, and implement solutions
+ Utilize knowledge of business processes, system processes, and industry standards to solve complex issues
+ Analyze information and make evaluative judgements to recommend solutions and improvements
+ Conduct testing and debugging, utilize script tools, and write basic code for design specifications
+ Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures
+ Develop working knowledge of Citi's information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Additional Job Description
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing data wrangling, scarping, cleaning using both Java or Python
Strong experience on data structure.
Extensively work on API integration.
- Monitoring performance and advising any necessary infrastructure changes
- Defining data retention policies
Skills and Qualifications
- Proficient understanding of distributed computing principles
- Proficient in Java or Pyhton and some part of machine learning
- Proficiency with Hadoop v2, MapReduce, HDFS,Pyspark,Spark
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Spark
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of various ETL techniques and frameworks, such as Flume
- Experience with various messaging systems, such as Kafka or RabbitMQ
- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
- Experience with Cloudera/MapR/Hortonworks
**Qualifications:**
+ 0-2 years of relevant experience
+ Experience in programming/debugging used in business applications
+ Working knowledge of industry practice and standards
+ Comprehensive knowledge of specific business area for application development
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
Big Data Engineer
Posted 3 days ago
Job Viewed
Job Description
Pay Range: 15 - 25 LPA (INR)
Required Qualifications:
- 3–5 years of hands-on experience in Big Data engineering.
- Proficiency in Python (Java is a plus).
- Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus.
- Solid problem-solving skills and a structured thought process.
- Experience with data migration and cloud-native development is highly desirable.
- Experience with building RESTful APIs
- Thorough understanding of JSON, and data structure fundamentals
- Expertise in objected oriented analysis and design across a variety of platforms
- Fundamental knowledge of distributed computing and Experience in Big Data Technologies like Spark, Hive, distributed computing.
- Experience with workflow scheduling tools. Airflow experience is desirable.
- Demonstrated experience in Agile development, application design, software development, and testing
- Hands-on expertise with application design, software development and automated testing
- Experience with distributed (multi-tiered) systems, algorithms, and relational databases
- Aptitude for learning and applying programming concepts
- Ability to effectively communicate with internal and external business partners
Nice to Have:
- Experience working in hybrid teams and agile environments.
- Exposure to GenAI or AI/ML-based applications.
- Familiarity with vendor collaboration (e.g., Infosys, IBM, CDS).
About the Role:
Insight Global is seeking a skilled and motivated Big Data Engineer to join our clients GenAI applications team based out of India. This role is part of a strategic initiative to migrate their data infrastructure from on-premise to Google Cloud Platform (GCP). You will work closely with a high-performing team of engineers and leaders to build scalable data solutions and support cloud-native application development.
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
We’re looking for a skilled Big Data Engineer.
Role Highlights:
Position: Big Data Engineer
Experience: 4+ years
Location- Remote
Work mode- WFH
Notice Period: Immediate/15 days joiners Mandatory.
Key Skills:-
Big Data, AWS, CI/CD Pipelines, Scala , Python or Java or C++
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
Greetings from TCS!
TCS is hiring for Big Data (PySpark)
Location: - Chennai/Bangalore
Desired Experience Range: 3 + Years
Must-Have
• PySpark • Hive
Good-to-Have
• Spark • HBase • DQ tool • Agile Scrum experience • Exposure in data ingestion from disparate sources onto Big Data platform
Thanks
Anshika
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
Job Title: Big Data Engineer
Location: Gurugram
Experience: 4-8 Years
Notice Period: Immediate to 15 Days
Key Responsibilities:
- Design and develop scalable Big Data solutions using PySpark and Hive.
- Perform Spark job optimization for performance tuning and efficient processing.
- Work with large-scale data pipelines and ensure smooth data ingestion and transformation.
- Collaborate with data analysts and engineering teams to implement data solutions.
Required Skills:
- Strong hands-on experience with Big Data technologies , especially PySpark and Hive .
- Proven expertise in Spark optimization and performance tuning.
Good to Have:
- Exposure to Google Cloud Platform (GCP) or other cloud-based data platforms.
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
Greetings from TCS!
Position: Big Data Engineer
Job Location: Pune
Experience: 5+
Job Title: Big Data Engineer
Must Have: Pyspark ,Scala , Python
Good To Have: Scala / Java
Responsibilities of the job:
- Prepare High Level and Detail Design based on requirements for development team to progress
- Work with Architect, Tech Leads and business for requirement/solution design clarifications
- Provide estimate for effort and timeline for application development
- Provide inputs for test strategy and planning
- Clearly articulate merits and pitfalls in different solution approaches
- Work with Development teams to enable delivery of solution as per the architecture/design by frequent reviews and providing clarifications and technical support required to team
- Support testing, deployment and implementation
- Communicating in a timely manner any potential roadblocks and impediments which may impact development effort
- Collaborate with other development and support teams if required
- Take ownership of code development till production implementation
Work Location
- India: Pune
TCS Eligibility Criteria:
*BE/B.tech/MCA/M.Sc./MS with minimum 3 years of relevant IT-experience post Qualification.
*Only Full-Time courses would be considered.
*Candidates who have attended TCS interview within 1 month need not apply.
Referrals are always welcome!
Jerin L Varghese
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
We are intending to hire Data engineer to handle day-to-day activities involving data ingestion from multiple source locations, help identify data sources, to troubleshoot issues, and engage with a third-party vendor to meet stakeholders’ needs.
Work Location: Chennai or Hyderabad or Pune
WFO.
Shift hours: 2.00pm to 11.00pm IST.
Required Immediate Joiners.
Required Skills :
- Python
- Processing of large quantities of text documents
- Extraction of text from Office and PDF documents
- Input json to an API, output json to an API
- Nifi (or similar technology compatible with current EMIT practices)
- Basic understanding of AI/ML concepts
- Database/Search engine/SOLR skills
- SQL – build queries to analyze, create and update databases
- Understands the basics of hybrid search
- Experience working with terabytes (TB) of data
- Basic OpenML/Python/Azure knowledge
- Scripting knowledge/experience in an Azure environment to automate
- Cloud systems experience related to search and databases
Platforms:
- DataBricks
- Snowflake
- ESRI ArcGIS / SDE
- New GenAI app being developed
Scope of work :
1. Ingest TB of data from multiple sources identified by the Ingestion Lead
2. Optimize data pipelines to improve on data processing, speed, and data availability
4. Make data available for end users from several hundred LAN and SharePoint areas
5. Monitor data pipelines daily and fix issues related to scripts, platforms, and ingestion
6. Work closely with the Ingestion Lead & Vendor on issues related to data ingestion
Technical Skills demonstrated:
1. SOLR - Backend database
2. Nifi - Data movement
3. Pyspark - Data Processing
4. Hive & Oozie - For jobs monitoring
5. Querying - SQL, HQl and SOLR querying
6. SQL
7. Python
Behavioral Skills demonstrated:
1. Excellent communication skills
2. Ability to receive direction from a Lead and implement
3. Prior experience working in an Agile setup, preferred
4. Experience troubleshooting technical issues and quality control checking of work
5. Experience working with a globally distributed team in different
Be The First To Know
About the latest Big data Jobs in India !
Big Data Engineer
Posted 8 days ago
Job Viewed
Job Description
Immediate #HIRING for a highly motivated and experienced Big Data Engineer to join our growing team.
We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies.
If you're passionate about building robust applications and thrive in a dynamic environment, please share your resume at
Job Title: Big Data Engineer
Experience: 5–9 Years
Location: Hyderabad-Hybrid
Employment Type: Full-Time
Job Summary:
We are seeking a skilled Big Data Engineer with 5–9 years of experience in building and managing scalable data pipelines and analytics solutions. The ideal candidate will have strong expertise in Big Data, Hadoop, Apache Spark, SQL, Hadoop, and Data Lake/Data Warehouse architectures. Experience working with any cloud platform (AWS, Azure, or GCP) is preferred. Top Universities or colleges prefeered.
Big Data Engineer
Posted 3 days ago
Job Viewed
Job Description
Required Qualifications:
3–5 years of hands-on experience in Big Data engineering.
Proficiency in Python (Java is a plus).
Strong understanding of Google Cloud Platform (GCP) services and architecture. GCP certification is a plus.
Solid problem-solving skills and a structured thought process.
Experience with data migration and cloud-native development is highly desirable.
Experience with building RESTful APIs
Thorough understanding of JSON, and data structure fundamentals
Expertise in objected oriented analysis and design across a variety of platforms
Fundamental knowledge of distributed computing and Experience in Big Data Technologies like Spark, Hive, distributed computing.
Experience with workflow scheduling tools. Airflow experience is desirable.
Demonstrated experience in Agile development, application design, software development, and testing
Hands-on expertise with application design, software development and automated testing
Experience with distributed (multi-tiered) systems, algorithms, and relational databases
Aptitude for learning and applying programming concepts
Ability to effectively communicate with internal and external business partners
Nice to Have:
Experience working in hybrid teams and agile environments.
Exposure to GenAI or AI/ML-based applications.
Familiarity with vendor collaboration (e.g., Infosys, IBM, CDS).
About the Role:
Insight Global is seeking a skilled and motivated Big Data Engineer to join our clients GenAI applications team based out of India. This role is part of a strategic initiative to migrate their data infrastructure from on-premise to Google Cloud Platform (GCP). You will work closely with a high-performing team of engineers and leaders to build scalable data solutions and support cloud-native application development.
Big Data Engineer
Posted 5 days ago
Job Viewed
Job Description
Work Location: Chennai or Hyderabad or Pune
WFO.
Shift hours: 2.00pm to 11.00pm IST.
Required Immediate Joiners.
Required Skills :
Python
Processing of large quantities of text documents
Extraction of text from Office and PDF documents
Input json to an API, output json to an API
Nifi (or similar technology compatible with current EMIT practices)
Basic understanding of AI/ML concepts
Database/Search engine/SOLR skills
SQL – build queries to analyze, create and update databases
Understands the basics of hybrid search
Experience working with terabytes (TB) of data
Basic OpenML/Python/Azure knowledge
Scripting knowledge/experience in an Azure environment to automate
Cloud systems experience related to search and databases
Platforms:
DataBricks
Snowflake
ESRI ArcGIS / SDE
New GenAI app being developed
Scope of work :
1. Ingest TB of data from multiple sources identified by the Ingestion Lead
2. Optimize data pipelines to improve on data processing, speed, and data availability
4. Make data available for end users from several hundred LAN and SharePoint areas
5. Monitor data pipelines daily and fix issues related to scripts, platforms, and ingestion
6. Work closely with the Ingestion Lead & Vendor on issues related to data ingestion
Technical Skills demonstrated:
1. SOLR - Backend database
2. Nifi - Data movement
3. Pyspark - Data Processing
4. Hive & Oozie - For jobs monitoring
5. Querying - SQL, HQl and SOLR querying
6. SQL
7. Python
Behavioral Skills demonstrated:
1. Excellent communication skills
2. Ability to receive direction from a Lead and implement
3. Prior experience working in an Agile setup, preferred
4. Experience troubleshooting technical issues and quality control checking of work
5. Experience working with a globally distributed team in different