!["modules" : [
{
"file" : "lib/Path/Extended.pm",
"in_basedir" : 0,
"in_lib" : 1,
"module" : "Path::Extended",
"uses" : {
"Sub::Install" : 1,
"strict" : 1,
"warnings" : 1
}
}
]](https://image.slidesharecdn.com/yapceu2012-120822051842-phpapp02/85/CPANTS-Kwalitative-website-and-its-tools-48-320.jpg)
![use WorePAN;
my $worepan = WorePAN->new(
root => 'path/to/a/directory/',
files => [qw(
I/IS/ISHIGAKI/WorePAN-0.01.tar.gz
)],
use_backpan => 1,
no_network => 0,
cleanup => 1,
);](https://image.slidesharecdn.com/yapceu2012-120822051842-phpapp02/85/CPANTS-Kwalitative-website-and-its-tools-69-320.jpg)
![use WorePAN;
my $worepan = WorePAN->new(
root => 'path/to/a/directory/',
files => [qw(
I/IS/ISHIGAKI/WorePAN-0.01.tar.gz
)],
local_mirror => '/home/ishigaki/minicpan/',
no_network => 1,
cleanup => 1,
);](https://image.slidesharecdn.com/yapceu2012-120822051842-phpapp02/85/CPANTS-Kwalitative-website-and-its-tools-70-320.jpg)
![Error is not a stash.
"error" : {
"easily_repackageable" : "easily_repackageable_by_fedora",
"easily_repackageable_by_fedora" : "fits_fedora_license",
"metayml_conforms_spec_current" : [
"1.4",
"Expected a map structure from data string or file. [Validation: 1.4]"
],
"metayml_conforms_to_known_spec" : [
"1.0",
"Expected a map structure from data string or file. [Validation: 1.0]"
],
"no_pod_errors" : " home cpants tmp analyze 11442 8001be43fb65..."
}](https://image.slidesharecdn.com/yapceu2012-120822051842-phpapp02/85/CPANTS-Kwalitative-website-and-its-tools-85-320.jpg)
CPANTS: Kwalitative website and its tools
- 1. CPANTS Kwalitative website and its tools Kenichi Ishigaki (charsbar) @YAPC::EU 2012 August 22, 2012
- 2. Kenichi Ishigaki (charsbar) From Shibuya.pm, Tokyo, Japan.
- 3. Freelancer - Perl programmer - Writer/Translator
- 4. Around 40 CPAN distributions
- 5. DBD::SQLite
- 7. We have been enjoying the CPANTS game since 2005.
- 8. 輝け！全日本最強 CPAN Author 決定選手権 by Koichi Taniguchi http://blog.livedoor.jp/nipotan/archives/16108466.html
- 9. He picked up Japanese authors by eye.
- 10. Our names are easy to find.
- 11. There were not so many authors. - Total: ~4000 - Japanese: ~50
- 12. YAPC::Asia increased the number of Japanese authors.
- 13. YAPC::Asia / Japanese authors 2006 (Mar) 98 2007 (Apr) 154 2008 (May) 191 2009 (Sep) 228 2010 (Oct) 255 2011 (Oct) 270
- 14. Needed something to pick up Japanese authors more easily.
- 15. That's why I created a list of Japanese authors and a script to maintain it.
- 16. I've been reporting the Japanese top 10 authors since 2008.
- 17. I've been adding something new every year.
- 18. 2008: sum of the kwalitee scores per author
- 19. 2009: authors who released most in the year
- 21. 2011: launched a website (finally) acme.cpanauthors.org
- 23. It had one big problem.
- 24. No data.
- 25. The official CPANTS site had been down for some time.
- 26. I needed to set up mine.
- 27. I created a private repository and put everything into it.
- 28. Merged recent commits from domm's repository.
- 29. Added a few columns.
- 30. Tweaked Catalyst/DBIC stuff.
- 31. It worked.
- 32. Warnings were left.
- 33. I needed to find some tuits to remove them.
- 36. Warnings were removed.
- 37. Ported some of the changes I did locally to daxim's repository.
- 38. Showed a new acme.cpanauthors.org featuring CPANTS info.
- 42. Unfortunately, the porting took too much time.
- 43. I didn't merge the changes back to my repository.
- 44. OSDC.TW
- 45. I finally merged the changes.
- 46. Got several reports that CPANTS was broken.
- 47. What broke CPANTS was a small change.
- 48. "modules" : [ { "file" : "lib/Path/Extended.pm", "in_basedir" : 0, "in_lib" : 1, "module" : "Path::Extended", "uses" : { "Sub::Install" : 1, "strict" : 1, "warnings" : 1 } } ]
- 49. I don't think this change is bad.
- 50. Module::CPANTS:: ProcessCPAN shouldn't have died by this.
- 51. It should have had tests.
- 52. Is should have run faster.
- 53. It should have been easier to fix analysis.
- 54. Enough issues for a summer.
- 55. What should we do?
- 56. - We need tests. - we need to find test cases. - we need to do it many times.
- 57. Making it run faster is the first priority.
- 58. I wrote a barebone script to store data in parallel.
- 59. JSON create table if not exists analysis ( id integer primary key autoincrement, path text unique, distv text, author text, json text, duration integer );
- 62. SQLite queue
- 63. Beware a race condition my ($id) = $dbh->selectrow_array(" SELECT id FROM queue WHERE status = 0 LIMIT = 1 "); $dbh->do(" UPDATE queue SET status = 1 WHERE id = ? ", undef, $id);
- 64. sqlite_update_hook my $id; my $dbh->sqlite_update_hook(sub { (undef, undef, undef, $id) = @_; });
- 65. $dbh->do(" UPDATE queue SET status = 1, WHERE id IN ( SELECT id FROM queue WHERE status = 0 LIMIT 1 ) ");
- 68. WorePAN - Bundling is bad - We need a specific version - Derived from OrePAN
- 69. use WorePAN; my $worepan = WorePAN->new( root => 'path/to/a/directory/', files => [qw( I/IS/ISHIGAKI/WorePAN-0.01.tar.gz )], use_backpan => 1, no_network => 0, cleanup => 1, );
- 70. use WorePAN; my $worepan = WorePAN->new( root => 'path/to/a/directory/', files => [qw( I/IS/ISHIGAKI/WorePAN-0.01.tar.gz )], local_mirror => '/home/ishigaki/minicpan/', no_network => 1, cleanup => 1, );
- 71. use WorePAN; my $worepan = WorePAN->new( root => 'path/to/a/directory/', dists => { 'Catalyst-Runtime' => 5.9, 'DBIx-Class' => 0, }, cleanup => 1, );
- 72. Bonus features my $worepan = WorePAN->new( root => 'path/to/a/CPAN/mirror/', cleanup => 0, ); my $authors = $worepan->authors; my $modules = $worepan->modules; my $file = $worepan->files; my $dists = $worepan->latest_distributions;
- 74. Now we have enough tools.
- 75. Processing time is significantly decreased.
- 76. What's next?
- 78. I'm preparing the data now.
- 80. Merging information from external sources. - CPAN indices - CPAN uploads database
- 81. Calculating scores on prerequisite modules.
- 82. It will be this year's something new in my annual report.
- 83. And then, I'll move on to fixing the metrics.
- 84. Some of them are badly broken. "versions" : { "lib/Data/Phrasebook.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Debug.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Generic.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Loader.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Loader/Base.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Loader/Text.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/Plain.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/SQL.pm" : "use vars qw($VERSION);¥n", "lib/Data/Phrasebook/SQL/Query.pm" : "use vars qw($VERSION);¥n" },
- 85. Error is not a stash. "error" : { "easily_repackageable" : "easily_repackageable_by_fedora", "easily_repackageable_by_fedora" : "fits_fedora_license", "metayml_conforms_spec_current" : [ "1.4", "Expected a map structure from data string or file. [Validation: 1.4]" ], "metayml_conforms_to_known_spec" : [ "1.0", "Expected a map structure from data string or file. [Validation: 1.0]" ], "no_pod_errors" : " home cpants tmp analyze 11442 8001be43fb65..." }
- 86. Should have initialize/finalize phases. Module::CPANTS::Kwalitee::Distros doesn't clean up after mirrored Debian CPANTS file https://rt.cpan.org/Ticket/Display.html?id=51514
- 87. There are much more to do. - JSON API for metacpan.org and so on. - Email Reporting like CPAN Testers - Evaluate new Kwalitee indicators - New metrics like portable filename - Blog about recent tendency - More comprehensive tests - Analysis per perl version/architecture - Cover Perl::Critic, CPAN::Critic::Module::Abstract - 35 RT tickets and several github isses
- 88. Resources github.com/charsbar/www-cpants github.com/charsbar/worepan github.com/daxim/Module-CPANTS-Analyse
- 89. Questions?
- 90. Thank you