Quality taxonomies
RQ1 describes generated-code outcomes. RQ2 describes properties of training samples and corpora. Coding is multi-label: a paper can support several headings, once per heading. Counts indicate research attention and do not measure defect prevalence.
RQ1: Generated-code quality
| Dimension | Immediate failure condition |
|---|---|
| Correctness | Syntax, logic, API, or dependency failures under task-required conditions |
| Security | Weaknesses permitting unauthorized access, disclosure, alteration, or unsafe execution |
| Compliance | Licensing, privacy, harmful-content, or other rights-related problems |
| Robustness | Failure to tolerate abnormal inputs, boundary cases, or runtime disruption |
| Maintainability | Structure or coupling that makes modification and extension difficult |
| Understandability | Naming, comments, or expressions that obscure intent |
| Efficiency | Excessive execution time or memory for the required behavior |
| Generation Derailment | Repetition, truncation, verbosity, or deviation from generation instructions |
The immediate failure condition determines the heading when manifestations overlap. A redundant structure that impedes modification concerns Maintainability; repetitive or instruction-divergent output concerns Generation Derailment.
The 195 paper–dimension memberships involve 98 papers. Correctness has the largest literature coverage. The codebook gives operational definitions and inclusion/exclusion boundaries.
RQ2: Training-data quality
Code attributes comprise Correctness, Security, Compliance, Robustness, Maintainability, Understandability, and Efficiency. They concern the code sample itself.
Non-code and corpus attributes comprise textual Compliance and Security Risks, Distribution Imbalance, Redundancy, Inadequate Diversity, Data Contamination Risks, and Low-Value Data. They concern surrounding content, supervision, composition, or training–evaluation overlap.
The 165 paper–heading memberships involve 78 papers. Distribution imbalance concerns relative frequency among represented strata; inadequate diversity concerns absent or effectively uncovered strata. Redundancy concerns repetition within training data; contamination concerns overlap with evaluation material.
Supporting papers
Browse 106 distinct papers by quality category. Expand a category to see its supporting papers, or select a linked category above to open it directly. Membership is multi-label: a paper appears in every category it supports, so category counts should not be added to obtain a distinct-paper total. Stable study IDs link each entry to the complete 114-paper register.
RQ1 · Generated-code quality
Correctness 74 papers
Generated code fails to implement intended semantics or to execute validly.
LLMs Meet Library Evolution
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
Qwen
Qwen Technical Report
Qwen2
Qwen2 Technical Report
Phi-4
Phi-4 Technical Report
Copilot Evaluation
An Empirical Evaluation of GitHub Copilot’s Code Suggestions
HalluCode
Exploring and Evaluating Hallucinations in LLM-Powered Code Generation
CodeHalu
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
Mercury
Mercury: A Code Efficiency Benchmark for Code Large Language Models
SStuBs
Large Language Models and Simple, Stupid Bugs
package hallucinations
We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs
HallTrigger
Code Hallucination
Large Language Models for Code
Large Language Models for Code: Security Hardening and Adversarial Testing
The Counterfeit Conundrum
The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?
Bugs in LLM-generated Code
Bugs in Large Language Models Generated Code: An Empirical Stud
GitHub Copilot, Amazon CodeWhisperer, ChatGPT
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
ChatGPT Code Quality
No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
CloudAPIBench
On Mitigating Code LLM Hallucinations with API Documentation
CodeMirage
CodeMirage: Hallucinations in Code Generated by Large Language Models
AutoAPIEval
A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
When Fine-Tuning LLMs Meets Data Privacy
When Fine-Tuning LLMs Meets Data Privacy: An Empirical Study of Federated Learning in LLM-Based Program Repair
From Effectiveness to Efficiency
From Effectiveness to Efficiency: Comparative Evaluation of Code Generated by LCGMs for Bilingual Programming Questions
Software Librarian
Is ChatGPT a Good Software Librarian? An Exploratory Study on the Use of ChatGPT for Software Library Recommendations
Codequal Analyzer
Improving LLM-Generated Code Quality with GRPO
Artificial-Intelligence Generated Code Considered Harmful
Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation
Unveiling Inefficiencies in LLM-Generated Code
Unveiling Inefficiencies in LLM-Generated Code: Toward a Comprehensive Taxonomy
Python Tests Quality
Quality Assessment of Python Tests Generated by Large Language Models
CoQuIR
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval
REAL
Training Language Models to Generate Quality Code with Program Analysis Feedback
CIDRe
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
SwallowCode
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Refining ChatGPT-Generated Code
Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues
ReCode
ReCode: Updating Code API Knowledge with Reinforcement Learning
Data-efficient Fine-tuning
Data-efficient LLM Fine-tuning for Code Generation
CRPE
CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
DeepSeek-Coder
DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
Code Pretraining
How Does Code Pretraining Affect Language Model Task Performance?
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Every Sample Matters
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
Code Data Training Stage
At Which Training Stage Does Code Data Help LLMs Reasoning?
WaveCoder
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
Brevity is the soul of wit
Brevity is the soul of wit: Pruning long files for code generation
Benchmark Builders
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Generated Code Diversity
Is Functional Correctness Enough to Evaluate Code Language Models? Exploring Diversity of Generated Codes
RedStone
RedStone: Curating General, Code, Math, and QA Data for Large Language Models
Code Llama
Code Llama: Open Foundation Models for Code
Codex
Evaluating Large Language Models Trained on Code
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
CODEJUDGE
CODEJUDGE : Evaluating Code Generation with Large Language Models
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
Unseen Horizons
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
Imperfect Code Generation
Imperfect Code Generation: Uncovering Weaknesses in Automatic Code Generation by Large Language Models
LLM-ProS
LLM-ProS: Analyzing Large Language Models’ Performance in Competitive Problem Solving
ClassEval
Evaluating Large Language Models in Class-Level Code Generation
UCD-Training
Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs
RealSec-Bench
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
ShortCoder
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code GenerationPreprint
APIKG4SYN
Framework-Aware Code Generation with API Knowledge Graph-Constructed Data: A Study on HarmonyOS
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Adadec
Adadec: Uncertainty-guided adaptive decoding for llm-based code generation
AllianceCoder
What to retrieve for effective retrieval-augmented code generation? an empirical study and beyond
RustEvo^ 2
RustEvo^ 2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
Llm Hallucinations in Practical Code Generation
Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Security 27 papers
Generated code contains weaknesses permitting unauthorized access disclosure alteration or unsafe execution.
Copilot Security
Is GitHub’s Copilot as Bad as Humans at Introducing Vulnerabilities in Code?
Large Language Models for Code
Large Language Models for Code: Security Hardening and Adversarial Testing
Purple Llama CYBERSECEVAL
Purple Llama CYBERSECEVAL: A Secure Coding Benchmark for Language Models
Lost at C
Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants
AI Assistants Security
Do Users Write More Insecure Code with AI Assistants?
GitHub Copilot, Amazon CodeWhisperer, ChatGPT
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
ChatGPT Code Quality
No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
CodeMirage
CodeMirage: Hallucinations in Code Generated by Large Language Models
DeVAIC
DeVAIC: A Tool for Security Assessment of AI-generated Code
PTMs
Comparing Robustness Against Adversarial Attacks in Code Generation: LLM-Generated vs. Human-Written
Software Librarian
Is ChatGPT a Good Software Librarian? An Exploratory Study on the Use of ChatGPT for Software Library Recommendations
Codequal Analyzer
Improving LLM-Generated Code Quality with GRPO
Artificial-Intelligence Generated Code Considered Harmful
Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation
CoQuIR
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval
REAL
Training Language Models to Generate Quality Code with Program Analysis Feedback
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
Codex
Evaluating Large Language Models Trained on Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
Unseen Horizons
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
RealSec-Bench
RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Llm Hallucinations in Practical Code Generation
Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Compliance 14 papers
Generated code violates legal ethical or regulatory constraints.
AI Assistants Security
Do Users Write More Insecure Code with AI Assistants?
DeSec
Decoding Secret Memorization in Code LLMs Through Token-Level Characterization
When Fine-Tuning LLMs Meets Data Privacy
When Fine-Tuning LLMs Meets Data Privacy: An Empirical Study of Federated Learning in LLM-Based Program Repair
Bias Unveiled
Bias Unveiled: Investigating Social Bias in LLM-Generated Code
FairCoder
FairCoder: Evaluating Social Bias of LLMs in Code Generation
CodeIP
CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code
CodeMI
Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach
CodeCipher
CodeCipher: Learning to Obfuscate Source Code Against LLMs
Code Llama
Code Llama: Open Foundation Models for Code
Codex
Evaluating Large Language Models Trained on Code
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
Code Generation Survey
A Survey on Large Language Models for Code Generation
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Robustness 11 papers
Generated code cannot remain stable under abnormal inputs boundary conditions or non-malicious runtime disruptions.
Bugs in LLM-generated Code
Bugs in Large Language Models Generated Code: An Empirical Stud
GitHub Copilot, Amazon CodeWhisperer, ChatGPT
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
CloudAPIBench
On Mitigating Code LLM Hallucinations with API Documentation
Codequal Analyzer
Improving LLM-Generated Code Quality with GRPO
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
CODEJUDGE
CODEJUDGE : Evaluating Code Generation with Large Language Models
Unseen Horizons
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Beyond Functional Correctness
Beyond functional correctness: Investigating coding style inconsistencies in large language models
RobGen
A Preliminary Study on the Robustness of Code Generation by Large Language Models
Maintainability 23 papers
Generated code is difficult to modify extend or reuse without introducing risk.
Copilot Evaluation
An Empirical Evaluation of GitHub Copilot’s Code Suggestions
HallTrigger
Code Hallucination
GitHub Copilot, Amazon CodeWhisperer, ChatGPT
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
CodeMirage
CodeMirage: Hallucinations in Code Generated by Large Language Models
Codequal Analyzer
Improving LLM-Generated Code Quality with GRPO
Unveiling Inefficiencies in LLM-Generated Code
Unveiling Inefficiencies in LLM-Generated Code: Toward a Comprehensive Taxonomy
Python Tests Quality
Quality Assessment of Python Tests Generated by Large Language Models
CoQuIR
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval
REAL
Training Language Models to Generate Quality Code with Program Analysis Feedback
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
ROSE
ROSE: Transformer-Based Refactoring Recommendation for Architectural Smells
Refining ChatGPT-Generated Code
Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues
Seed-Coder
Seed-Coder: Let the Code Model Curate Data for Itself
CodeSmellEval
How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
CODEJUDGE
CODEJUDGE : Evaluating Code Generation with Large Language Models
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Beyond Functional Correctness
Beyond functional correctness: Investigating coding style inconsistencies in large language models
Understandability 13 papers
Humans have difficulty comprehending the generated code's logic intent or structure.
Unveiling Inefficiencies in LLM-Generated Code
Unveiling Inefficiencies in LLM-Generated Code: Toward a Comprehensive Taxonomy
CIDRe
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
Seed-Coder
Seed-Coder: Let the Code Model Curate Data for Itself
CRPE
CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
CodeSmellEval
How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
ShortCoder
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code GenerationPreprint
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Beyond Functional Correctness
Beyond functional correctness: Investigating coding style inconsistencies in large language models
Efficiency 22 papers
Generated code unnecessarily consumes time or memory resources.
CodeHalu
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
EffiBench
EffiBench: Benchmarking the Efficiency of Automatically Generated Code
Mercury
Mercury: A Code Efficiency Benchmark for Code Large Language Models
GitHub Copilot, Amazon CodeWhisperer, ChatGPT
Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT
ChatGPT Code Quality
No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
LLM-generated Code Efficiency
On Evaluating the Efficiency of Source Code Generated by LLMs
From Effectiveness to Efficiency
From Effectiveness to Efficiency: Comparative Evaluation of Code Generated by LCGMs for Bilingual Programming Questions
ENAMEL
How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark
Codequal Analyzer
Improving LLM-Generated Code Quality with GRPO
Unveiling Inefficiencies in LLM-Generated Code
Unveiling Inefficiencies in LLM-Generated Code: Toward a Comprehensive Taxonomy
CoQuIR
CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
Data-efficient Fine-tuning
Data-efficient LLM Fine-tuning for Code Generation
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
CODEJUDGE
CODEJUDGE : Evaluating Code Generation with Large Language Models
Unseen Horizons
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Beyond Functional Correctness
Beyond functional correctness: Investigating coding style inconsistencies in large language models
COFFE
COFFE: A Code Efficiency Benchmark for Code Generation
Generation Derailment 11 papers
The generated response departs from the task-directed course needed to provide a usable code deliverable.
HalluCode
Exploring and Evaluating Hallucinations in LLM-Powered Code Generation
HallTrigger
Code Hallucination
Bugs in LLM-generated Code
Bugs in Large Language Models Generated Code: An Empirical Stud
RPG
Rethinking Repetition Problems of LLMs in Code Generation
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Code Llama
Code Llama: Open Foundation Models for Code
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
Imperfect Code Generation
Imperfect Code Generation: Uncovering Weaknesses in Automatic Code Generation by Large Language Models
DRAINCODE
DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context PoisoningPreprint
MultiCodeIF
A hierarchical and evolvable benchmark for fine-grained code instruction following with multi-turn feedback
Code Copycat Conundrum
Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation
RQ2 · Training-data quality
Code attributes
Correctness 21 papers
Training code samples contain syntax logical functionality API or dependency defects.
LLMs Meet Library Evolution
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
CodeHalu
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
SStuBs
Large Language Models and Simple, Stupid Bugs
package hallucinations
We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs
Bugs in LLM-generated Code
Bugs in Large Language Models Generated Code: An Empirical Stud
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
SwallowCode
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Code Pretraining
How Does Code Pretraining Affect Language Model Task Performance?
Every Sample Matters
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
UCD-Training
Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs
RustEvo^ 2
RustEvo^ 2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Security 13 papers
Training code samples contain vulnerabilities credentials or high-risk implementation patterns.
Copilot Security
Is GitHub’s Copilot as Bad as Humans at Introducing Vulnerabilities in Code?
Purple Llama CYBERSECEVAL
Purple Llama CYBERSECEVAL: A Secure Coding Benchmark for Language Models
Lost at C
Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants
AI Assistants Security
Do Users Write More Insecure Code with AI Assistants?
DeVAIC
DeVAIC: A Tool for Security Assessment of AI-generated Code
Artificial-Intelligence Generated Code Considered Harmful
Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Codex
Evaluating Large Language Models Trained on Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
Llm Hallucinations in Practical Code Generation
Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Compliance 6 papers
Training code samples violate legal ethical or regulatory requirements.
DeSec
Decoding Secret Memorization in Code LLMs Through Token-Level Characterization
CodeMI
Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
Code Generation Survey
A Survey on Large Language Models for Code Generation
Uncovering Pretraining Code in LLMs
Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach
Robustness 2 papers
Training code samples lack fault tolerance for abnormal inputs boundary cases or runtime variation.
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
Beyond Functional Correctness
Beyond functional correctness: Investigating coding style inconsistencies in large language models
Maintainability 8 papers
Training code samples have structures that are difficult to modify extend or reuse.
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
SwallowCode
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
Seed-Coder
Seed-Coder: Let the Code Model Curate Data for Itself
CodeSmellEval
How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
Cracks in The Stack
Cracks in The Stack: Hidden Vulnerabilities and Licensing Risks in LLM Pre-Training Datasets
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
Understandability 7 papers
Training code samples express logic or intent unclearly.
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
Seed-Coder
Seed-Coder: Let the Code Model Curate Data for Itself
Benchmark Builders
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
ShortCoder
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code GenerationPreprint
Efficiency 6 papers
Training code samples contain resource-wasting or algorithmically suboptimal implementations.
EffiBench
EffiBench: Benchmarking the Efficiency of Automatically Generated Code
Mercury
Mercury: A Code Efficiency Benchmark for Code Large Language Models
Quality In, Quality Out
Quality In, Quality Out: Investigating Training Data's Role in AI Code Generation
SwallowCode
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
RTL-Breaker
RTL-Breaker: Assessing the Security of LLMs Against Backdoor Attacks on HDL Code Generation
ShortCoder
ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code GenerationPreprint
Non-code and corpus attributes
Compliance and Security Risks (Textual) 5 papers
Non-code textual training content is harmful copyright-infringing or privacy-leaking.
CodeCipher
CodeCipher: Learning to Obfuscate Source Code Against LLMs
Codex
Evaluating Large Language Models Trained on Code
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
Code Generation Survey
A Survey on Large Language Models for Code Generation
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
Distribution Imbalance Issues 29 papers
The corpus is uneven across languages domains resources data types tasks or difficulty.
LLMs Meet Library Evolution
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion
DataMan
DataMan: Data Manager for Pre-training Large Language Models
Phi-4
Phi-4 Technical Report
EffiBench
EffiBench: Benchmarking the Efficiency of Automatically Generated Code
CloudAPIBench
On Mitigating Code LLM Hallucinations with API Documentation
AutoAPIEval
A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models
Unveiling Inefficiencies in LLM-Generated Code
Unveiling Inefficiencies in LLM-Generated Code: Toward a Comprehensive Taxonomy
CIDRe
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Data-efficient Fine-tuning
Data-efficient LLM Fine-tuning for Code Generation
CRPE
CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
Code Pretraining
How Does Code Pretraining Affect Language Model Task Performance?
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Code Data Training Stage
At Which Training Stage Does Code Data Help LLMs Reasoning?
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Code Pre-training Impact
To Code, or Not To Code? Exploring Impact of Code in Pre-training
Logical Inference Pre-training
Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance?
Code Llama
Code Llama: Open Foundation Models for Code
Codex
Evaluating Large Language Models Trained on Code
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
Training Data Extraction
Understanding Privacy Risks of Large Language Models in Japanese Based on Training Data Extraction Attacks
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
APIKG4SYN
Framework-Aware Code Generation with API Knowledge Graph-Constructed Data: A Study on HarmonyOS
AATK Benchmark
Asleep at the keyboard? assessing the security of github copilot's code contributions
Redundancy Issues 16 papers
The corpus contains duplicate near-duplicate or excessively repetitive samples.
DataMan
DataMan: Data Manager for Pre-training Large Language Models
CodeHalu
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
Data-efficient Fine-tuning
Data-efficient LLM Fine-tuning for Code Generation
Code Pretraining
How Does Code Pretraining Affect Language Model Task Performance?
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Repetition In Repetition Out
Repetition In Repetition Out: Towards Understanding Neural Text Degeneration from the Data Perspective
DataComp-LM
DataComp-LM: In search of the next generation of training sets for language models
Code Llama
Code Llama: Open Foundation Models for Code
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
Training Data Extraction
Understanding Privacy Risks of Large Language Models in Japanese Based on Training Data Extraction Attacks
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing
LLM-ProS
LLM-ProS: Analyzing Large Language Models’ Performance in Competitive Problem Solving
Code Copycat Conundrum
Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation
Inadequate Diversity 26 papers
The corpus insufficiently covers scenarios domains edge cases environments or solution forms.
Qwen
Qwen Technical Report
Qwen2
Qwen2 Technical Report
HalluCode
Exploring and Evaluating Hallucinations in LLM-Powered Code Generation
CodeHalu
CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification
package hallucinations
We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs
HallTrigger
Code Hallucination
ChatGPT Code Quality
No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT
When Fine-Tuning LLMs Meets Data Privacy
When Fine-Tuning LLMs Meets Data Privacy: An Empirical Study of Federated Learning in LLM-Based Program Repair
DeVAIC
DeVAIC: A Tool for Security Assessment of AI-generated Code
Software Librarian
Is ChatGPT a Good Software Librarian? An Exploratory Study on the Use of ChatGPT for Software Library Recommendations
Infinite-Instruct
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
Security and Quality in LLM-Generated Code
Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis
ReCode
ReCode: Updating Code API Knowledge with Reinforcement Learning
WaveCoder
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning
RedStone
RedStone: Curating General, Code, Math, and QA Data for Large Language Models
Code Llama
Code Llama: Open Foundation Models for Code
Path Planning Evaluation
Assessing LLM code generation quality through path planning tasks
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
Synthetic Data Generation
Synthetic Data Generation Using Large Language Models: Advances in Text and Code
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
Imperfect Code Generation
Imperfect Code Generation: Uncovering Weaknesses in Automatic Code Generation by Large Language Models
UCD-Training
Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs
APIKG4SYN
Framework-Aware Code Generation with API Knowledge Graph-Constructed Data: A Study on HarmonyOS
RustEvo^ 2
RustEvo^ 2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation
Llm Hallucinations in Practical Code Generation
Llm hallucinations in practical code generation: Phenomena, mechanism, and mitigation
Data Contamination Risks 7 papers
Training data overlap with evaluation benchmarks or otherwise leak test solutions.
Phi-4
Phi-4 Technical Report
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Beyond Correctness
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
DataComp-LM
DataComp-LM: In search of the next generation of training sets for language models
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
Unseen Horizons
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
LLM-ProS
LLM-ProS: Analyzing Large Language Models’ Performance in Competitive Problem Solving
Low-Value Data 19 papers
Training data contributes little useful learning signal or introduces noise and error.
Less is More
Less is More: On the Importance of Data Quality for Unit Test Generation
DataMan
DataMan: Data Manager for Pre-training Large Language Models
Phi-4
Phi-4 Technical Report
Large Language Models for Code
Large Language Models for Code: Security Hardening and Adversarial Testing
CIDRe
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
Data-efficient Fine-tuning
Data-efficient LLM Fine-tuning for Code Generation
CRPE
CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generation
DeepSeek-Coder
DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
StarCoder 2 and The Stack v2
StarCoder 2 and The Stack v2: The Next Generation
Every Sample Matters
Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
Brevity is the soul of wit
Brevity is the soul of wit: Pruning long files for code generation
Benchmark Builders
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
DataComp-LM
DataComp-LM: In search of the next generation of training sets for language models
Codex
Evaluating Large Language Models Trained on Code
Datasets for Large Language Models
Datasets for Large Language Models: A Comprehensive Survey
MG-Verilog
MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
Code Generation Survey
A Survey on Large Language Models for Code Generation
DataRecipe
DataRecipe --- How to Cook the Data for CodeLLM?
aiXcoder-7B
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing