4,879 Databricks Python jobs in India
Job No Longer Available
This position is no longer listed on WhatJobs. The employer may be reviewing applications, filled the role, or has removed the listing.
However, we have similar jobs available for you below.
Big Data Engineering Expert
Posted today
Job Viewed
Job Description
We are seeking a seasoned Big Data Engineer to join our team. As a key member of our engineering organization, you will play a pivotal role in designing, building, and maintaining scalable data infrastructure.
Your primary focus will be on architecting and implementing cutting-edge big data solutions using technologies such as Spark, Druid, Drill, and ClickHouse. You will work closely with cross-functional teams to ensure seamless integration with existing systems and applications.
To be successful in this role, you will need 8+ years of experience in Python/Java development, CI/CD infrastructure, and cloud architecture, specifically Terraform. Additionally, you should have in-depth knowledge of big data pipelines, graph databases, and software engineering principles.
A Master's degree or Ph.D. in Computer Science, Information Technology, Data Science, Artificial Intelligence, Machine Learning, or a related field is highly desirable.
In return for your expertise, we offer a challenging and rewarding environment that fosters growth, innovation, and collaboration. If you are passionate about leveraging data to drive business outcomes, we encourage you to apply.
- Design and implement high-performance big data architectures using Spark, Druid, Drill, and ClickHouse.
- Develop and maintain scalable data pipelines to support real-time analytics and reporting.
- Collaborate with cross-functional teams to integrate big data solutions with existing systems and applications.
- Stay up-to-date with emerging trends and technologies in big data and machine learning.
Big Data Engineering Expert
Posted today
Job Viewed
Job Description
About Us
We are a leading provider of technology-enabled transformation services, driving innovation across various sectors including healthcare, technology, communications, manufacturing, and finance.
Our expertise lies in delivering cutting-edge solutions that streamline product development, maximize platform ROI, and drive meaningful business outcomes.
About the Role
Senior Big Data Engineer: We seek an experienced professional with a strong background in Big Data analytics platforms.
Key Responsibilities
• Develop and maintain large-scale Big Data pipelines using Spark, Kafka, Glue, EMR, Hudi, Schema Registry, and Data Lineage.
• Design and implement efficient data processing systems utilizing Neo4j, Neptune, JanusGraph, and Dgraph graph databases.
Requirements
• 8+ years of experience in Python/Java, CI/CD, infrastructure & cloud, Terraform, and depth in Big Data analytics.
• Strong understanding of data processing, machine learning, and artificial intelligence concepts.
Preferred Qualifications
• Master's degree (M.Tech/MS) or Ph.D. in Computer Science, Information Technology, Data Science, Artificial Intelligence, Machine Learning, Software Engineering, or a related technical field.
Big Data Engineering Leader
Posted today
Job Viewed
Job Description
As a Senior Engineering Manager-Big Data, Generative AI you will be responsible for driving innovation in distributed systems large-scale data pipelines and ML solutions leading the development of groundbreaking products defining the future of AI-driven network management. You will shape next-gen networking experiences at the cutting edge of Generative AI Machine Learning Big Data and Cloud Computing.
The AI Core group pioneers platforms and solutions for Generative AI including AI Agents RAG Knowledge Bases Data Mining Anomaly Detection and LLM fine-tuning. These innovations power flagship products while enabling entirely new offerings.
Big Data Engineering Director
Posted today
Job Viewed
Job Description
We are seeking a seasoned technical leader to drive innovation in distributed systems and large-scale data pipelines.
This is an exceptional opportunity for an experienced engineering manager to shape the future of network intelligence, leveraging cutting-edge technologies like Generative AI, Big Data, and Cloud Computing.
The ideal candidate will have a strong track record of delivering high-performance, scalable systems and mentoring top talent.
Requirements
Big Data Engineering Lead
Posted today
Job Viewed
Job Description
Senior Data Engineer Role
We are seeking an experienced Senior Data Engineer to lead our data engineering team. In this role, you will be responsible for developing architectures that meet our clients' needs, working closely with cross-functional teams to source data and make it available for downstream consumption.
The successful candidate will have experience in building data pipelines using batch processing with Apache Spark or Hive query language, as well as streaming data pipelines using Apache Spark Structured Streaming or Apache Flink on Kafka & Delta Lake. They will also have knowledge of NOSQL databases, particularly Cosmos DB, Restful APIs, and GraphQL.
In addition to technical skills, the ideal candidate will be a strong communicator who can work effectively with stakeholders to ensure regular communication and escalation of dependencies and challenges (risks). They will also have the ability to debug, fine tune, and optimize large-scale data processing jobs.
This is a senior leadership position that requires excellent problem analysis skills, a good understanding of Azure Databricks, and experience with at least one Cloud Infra provider (Azure/AWS). If you have a passion for big data and enjoy leading high-performing teams, we encourage you to apply.
Big Data Engineer
Posted 1 day ago
Job Viewed
Job Description
**Responsibilities:**
+ Design, development of BigData applications/ pipelines using Spark, Scala, SQL, Pyspark, Python, Java
+ Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 4-8 years of experience in software development, building large scale distributed data processing systems or large-scale applications
+ Designing & developing Big Data solutions with at least one end to end implementation.
+ Strong Hands-on experience in following technical skills: Apache Spark, Scala/ Java, XML/ JSON/ Parquet/ Avro, SQL, Linux, Hadoop Ecosystem (HDFS, Spark, Impala, HIVE, HBASE etc.), Kafka.
+ Performance analysis, troubleshooting and issue resolution and Exposure to latest Cloudera offerings like Ozone, Iceberg.
+ Intermediate level experience in Applications Development role
+ Consistently demonstrates clear and concise written and verbal communication
+ Demonstrated problem-solving and decision-making skills
+ Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
Big Data Engineer
Posted 1 day ago
Job Viewed
Job Description
**Responsibilities:**
+ Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements
+ Identify and analyze issues, make recommendations, and implement solutions
+ Utilize knowledge of business processes, system processes, and industry standards to solve complex issues
+ Analyze information and make evaluative judgements to recommend solutions and improvements
+ Conduct testing and debugging, utilize script tools, and write basic code for design specifications
+ Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures
+ Develop working knowledge of Citi's information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 3 to 5 years of relevant experience
+ Experience in programming/debugging used in business applications
+ Working knowledge of industry practice and standards
+ Comprehensive knowledge of specific business area for application development
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Additional Job Description
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing data wrangling, scarping, cleaning using both Java or Python
Strong experience on data structure.
Skills and Qualifications
- Proficient understanding of distributed computing principles
- Proficient in Java or Python and some part of machine learning
- Proficiency with Hadoop v2, MapReduce, HDFS, Pyspark, Spark
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Spark
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of various ETL techniques and frameworks, such as Flume
- Experience with various messaging systems, such as Kafka or RabbitMQ
- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
- Experience with Cloudera/MapR/Hortonworks
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
Be The First To Know
About the latest Databricks python Jobs in India !
Big Data Developer
Posted today
Job Viewed
Job Description
We are seeking a Big Data Engineer with strong hands-on experience in Spark and AWS technologies. The ideal candidate should demonstrate a deep understanding of big data concepts, programming fundamentals, and the ability to solve complex problems related to scalability, failure handling, and optimization.
Key Responsibilities:
- Design, develop, and optimize big data pipelines using Spark on AWS .
- Implement scalable and fault-tolerant data processing solutions.
- Troubleshoot and resolve performance bottlenecks in big data workflows.
- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
- Write clean, efficient, and well-documented code following core programming principles.
- Continuously improve existing data infrastructure for better reliability and performance.
Required Skills & Experience:
- Strong practical experience with Apache Spark and big data ecosystems.
- Hands-on experience with AWS services relevant to big data (e.g., EMR, S3, Lambda).
- Solid understanding of core programming fundamentals, including Object-Oriented Programming (OOP) concepts.
- Proven problem-solving skills related to scaling, failure handling, and performance optimization in big data environments.
- Ability to explain not just what technologies are used, but why and how they work.
- Familiarity with common big data terms and best practices.
Big Data Developer
Posted today
Job Viewed
Job Description
Greetings from TCS!
TCS is hiring for Big Data
Location: - Chennai/Mumbai/Pune
Desired Experience Range: 5 to 12 years
Must-Have**
- Working experience of Hadoop, Hive SQL’s, Spark, Bigdata Eco System Tools.
- Should be able to tweak queries and work on performance enhancement.
- The candidate will be responsible for delivering code, setting up environment, connectivity, deploying the code in production after testing.
- The candidate should have strong functional and technical knowledge to deliver what is required and he/she should be well acquainted with Banking terminologies. Occasionally, the candidate may have to be responsible as a primary contact and/or driver for small to medium size projects.
- The candidate should have strong DevOps and Agile Development Framework knowledge.
Good-to-Have
- Preferable to have good technical knowledge on Cloud computing, AWS or Azure Cloud Services.
- Strong conceptual and creative problem-solving skills, ability to work with considerable ambiguity, ability to learn new and complex concepts quickly.
- Experience in working with teams in a complex organization involving multiple reporting lines
- Solid understanding of object-oriented programming and HDFS concepts
Thanks
Anshika