GitHub Top Developer Source Code: 1.3M+ Files for Code Intelligence
ronantakizawa/github-top-code ↗
The **GitHub Top Developer Source Code** dataset, authored by *ronantakizawa*, aggregates over **1.3 million source code files** contributed by the most highly ranked GitHub developers between 2015 and 2025. Spanning **80+ programming languages**—including Python, JavaScript, Rust, Go, C/C++, and Java—the collection deliberately excludes configuration and documentation files, focusing solely on executable code under permissive licenses (MIT, Apache‑2.0, BSD, ISC, etc.). Each entry is enriched with metadata such as repository star count, description, primary language, file path, and developer affiliation, enabling nuanced analyses of code quality and popularity.
The dataset is stored in **Parquet** format and is compatible with the Hugging Face *datasets* library as well as data‑processing frameworks like *dask*, *polars*, and *mlcroissant*. It is split into deterministic *train* (≈90%), *test* (≈5%), and *validation* (≈5%) partitions by repository hash, ensuring that files from the same repo never appear in multiple splits and thus preventing data leakage during model training or evaluation. The README provides straightforward Python snippets for loading the dataset and filtering by language, stars, or developer username.
Trending due to its scale, rich metadata, and focus on top‑tier developers, this dataset is ideal for training code‑generation models, building code‑search engines, and conducting empirical software‑engineering research. Its public availability under an MIT license further encourages community contributions and downstream applications in the software‑engineering AI ecosystem.
Project Ideas
- Fine‑tune a code‑completion model on the train split to improve language‑specific autocompletion for popular languages like Python and Rust.
- Create a semantic code‑search engine that indexes the "content" field together with repo stars to surface high‑quality snippets for a given query.
- Analyze the relationship between repository star count and coding style by aggregating metrics (e.g., line length, comment density) across developers.
- Build a cross‑language translation benchmark by pairing files with the same functionality but different "file_language" values within the same repository.
- Develop a developer‑centric dashboard that visualizes code contributions, language diversity, and company affiliation using the provided metadata.