Your Git repository is a treasure trove of data about your project. Every commit you make tells a part of the story of your project. Would it not be great to run some analysis on it to see hidden pattern or learn about the dynamics at work?
This post is part of my journey to learn Python. You can find the other parts of this series here. You find the code for this post in my PythonFriday repository on GitHub.
PyDriller
With PyDriller we can access our Git repositories with Python code. We do not need to know the implementation details of Git; instead, we use the high-level API of PyDriller to find the interesting parts.
You can install PyDriller with this command:
|
1 |
pip install PyDriller |
Traverse commits
PyDriller allows us to traverse all commits with the traverse_commits() method in our repository. This is a good starting point to learn PyDriller and explore your repository in code.
The Repository class can point to a local copy or a remote address. The Commit object we get back has fields for all the attributes of a Git commit (like author, commit date, hash, and so forth).
|
1 2 3 |
# for commit in Repository('https://github.com/jgraber/PythonFriday').traverse_commits(): for commit in Repository('..\..\PythonFriday').traverse_commits(): print(f"{commit.committer_date} - {commit.hash[0:8]} - {commit.author.name}") |
This produces an output like this:
2021-10-22 23:30:00+02:00 – c816a116 – Johnny Graber
2021-10-23 23:15:00+02:00 – 4f379c7a – Johnny Graber
2021-10-24 11:18:10+02:00 – 64ec912d – Johnny Graber
2021-10-25 15:37:05+02:00 – 0a9b1109 – Johnny Graber
A bit more interesting is the field modified_files, that offers us a list of the changed files in this commit:
|
1 2 3 4 |
for commit in Repository('..\..\PythonFriday').traverse_commits(): print(f"{commit.hash} - {commit.committer_date} - {commit.author.name} - {commit.msg}") for file in commit.modified_files: print(f' - {file.filename} has changed ({file.change_type.name})') |
The output lists the changed files and how they changed:
– 5867629bf749_add_created_date_with_default_value_to_.py has changed (ADD)
– cd496010379c_remove_constraints_for_examples.py has changed (ADD)
– dff2171f8b62_create_index_for_employee_last_name_and_.py has changed (ADD)
– e4887a8bedc7_add_index_to_publisher_name.py has changed (ADD)
– book.py has changed (MODIFY)
– employee.py has changed (MODIFY)
– publisher.py has changed (MODIFY)
How often changed a file?
The CommitsCount() class allows us to take a range of commits to count how often a file got changed:
|
1 2 3 4 5 6 |
from pydriller.metrics.process.commits_count import CommitsCount metric = CommitsCount(path_to_repo='..\..\PythonFriday', from_commit='42722d558b3175a0c60ba7e513b7786ae6dbb591', to_commit='36425461ed2e42883ee7935af79cb620218f88b2') files = metric.count() pp.pprint(files) |
This gives us a dictionary with the files and their number of changes back:
{‘SQLAlchemy\\Core\\sqlalchemy_core_crud.py’: 2,
‘SQLAlchemy\\Core\\sqlalchemy_core_filters.py’: 3,
‘SQLAlchemy\\Core\\sqlalchemy_core_relationships.py’: 1,
‘SQLAlchemy\\ORM\\sqlalchemy_orm_relationships.py’: 1}
We can use this information to plan the next refactoring session. Files that change often tend to require more changes in the future. Therefore, cleaning up those files makes the next changes a lot simpler.
How much changed in a file?
With the LinesCount() class we can use a range of commits and get a counter for all added and removed lines per file.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
from pydriller.metrics.process.lines_count import LinesCount metric = LinesCount(path_to_repo='..\..\PythonFriday', from_commit='42722d558b3175a0c60ba7e513b7786ae6dbb591', to_commit='36425461ed2e42883ee7935af79cb620218f88b2') added_count = metric.count_added() removed_count = metric.count_removed() print('Total lines added per file:') pp.pprint(added_count) print('Total lines removed per file:') pp.pprint(removed_count) |
As with CommitsCount() we get a dictionary with the file path as key and the number of lines as the value:
Total lines added per file:
{‘SQLAlchemy\\Core\\sqlalchemy_core_crud.py’: 6,
‘SQLAlchemy\\Core\\sqlalchemy_core_filters.py’: 223,
‘SQLAlchemy\\Core\\sqlalchemy_core_relationships.py’: 200,
‘SQLAlchemy\\ORM\\sqlalchemy_orm_relationships.py’: 65}
Total lines removed per file:
{‘SQLAlchemy\\Core\\sqlalchemy_core_crud.py’: 136,
‘SQLAlchemy\\Core\\sqlalchemy_core_filters.py’: 7,
‘SQLAlchemy\\Core\\sqlalchemy_core_relationships.py’: 0,
‘SQLAlchemy\\ORM\\sqlalchemy_orm_relationships.py’: 0}
More to explore
The official documentation of PyDriller has many more methods to offer to dig even deeper into your repository. If you think of a question you have about your repository you most likely find an answer there.
Conclusion
PyDriller is a great help when you want to look at your Git repository and extract interesting data. Try it if you want to analyse your repository.