Apache Livy has officially graduated to a Top-Level Project, offering a lightweight REST API that allows multiple apps and notebooks to seamlessly share a central Spark cluster without needing local client installations.
On 4 June 2026, Apache Livy officially announced the graduation of Apache Incubator to become a Top-Level Project (TLP). This project first entered the incubator in June 2017, completing its journey toward becoming a fully independent, open-source project.
Apache Livy is a service that allows applications to communicate to an Apache Spark cluster using a standard web interface (REST API). It allows users to run Spark jobs and code snippets, and view the results either instantly or later on. By removing the need for a local Spark client on application servers, Livy shifts Spark interaction to a standard REST API. This architectural change introduces two major operational benefits:
-
Concurrent Cluster Sharing: It enables multiple data science notebooks (such as Jupyter or Zeppelin), microservices, and individual users to securely and simultaneously interact with a single, centralised Spark cluster without resource conflicts.
-
Simplified App Integration: It eliminates the overhead of managing local Spark installations on external machines, dramatically streamlining how web, mobile, and enterprise applications connect directly to a Spark backend.
To graduate, the project had to prove it could maintain a healthy, active, and collaborative community. Now that it has graduated, Livy is no longer guided by mentors. Instead, it is completely self-managed by its own Project Management Committee (PMC), which will oversee all future software updates, security fixes, and community growth.
The announcement underscores the broader scale of the Apache Software Foundation, which acts as a 501(c)(3) non-profit organisation and the global home for over 290 open-source projects, including industry standards such as Apache Airflow, Camel, Groovy, Kafka, and Cassandra.
