Core Strength
One of the things to consider in the brave new world of Perl Lambda functions we've been exploring together over the last few days, is dependance cost. Every individual web endpoint is effectively its own bundle of dependencies. If we're not careful, the dependencies can add up quickly - for example, DateTime and its required dependencies alone adds 30MB+ of source code and support files. This is a problem when we've got a platform enforced hard limit of 50MB of decompressed files in our Lambda sourcecode zipfile - we either need to start splitting things out into layers, or slimming down.
One thing we should not overlook is that Perl already comes with a collection of awesome modules for doing the core tings (hence the name - the core distribution.) We should use more of those when we can! But how do we know if chosen module is available?
Module::CoreList is a CPAN module designed to solve this problem. It keeps detailed information on what core modules were included with the main Perl distribution for each release.
It's accessible on the web from http://corelist.rpee.be/, via the module interface, or most simply, via a command line tool.
Streching our Core Muscles
The command line tool has a bunch of really handy options - let's take tour and see what we can work out.
How about when a perl module was first released with perl?
shell$ corelist List::Util
Data for 2019-11-20
List::Util was first released with perl v5.7.3What was the date that went live?
shell$ corelist -r 5.7.3
Perl v5.7.3 was released on 2002-03-05According to the List::Util's documentation the any function has been available since version 1.33. What version of perl was that released with?
shell$ corelist -a List::Util
Data for 2019-11-20
List::Util was first released with perl v5.7.3
v5.7.3 1.06_00
v5.8.0 1.07_00
v5.8.1 1.13
v5.8.2 1.13
v5.8.3 1.13
v5.8.4 1.13
v5.8.5 1.14
v5.8.6 1.14
v5.8.7 1.14
v5.8.8 1.18
v5.8.9 1.19
v5.9.0 1.13
v5.9.1 1.13
v5.9.2 1.14
v5.9.3 1.18
v5.9.4 1.18
v5.9.5 1.19
v5.10.0 1.19
v5.10.1 1.21
v5.11.0 1.21
v5.11.1 1.21
v5.11.2 1.22
v5.11.3 1.22
v5.11.4 1.22
v5.11.5 1.22
v5.12.0 1.22
v5.12.1 1.22
v5.12.2 1.22
v5.12.3 1.22
v5.12.4 1.22
v5.12.5 1.22
v5.13.0 1.22
v5.13.1 1.23
v5.13.2 1.23
v5.13.3 1.23
v5.13.4 1.23
v5.13.5 1.23
v5.13.6 1.23
v5.13.7 1.23
v5.13.8 1.23
v5.13.9 1.23
v5.13.10 1.23
v5.13.11 1.23
v5.14.0 1.23
v5.14.1 1.23
v5.14.2 1.23
v5.14.3 1.23
v5.14.4 1.23
v5.15.0 1.23
v5.15.1 1.23
v5.15.2 1.23
v5.15.3 1.23
v5.15.4 1.23
v5.15.5 1.23
v5.15.6 1.23
v5.15.7 1.23
v5.15.8 1.23
v5.15.9 1.23
v5.16.0 1.23
v5.16.1 1.25
v5.16.2 1.25
v5.16.3 1.25
v5.17.0 1.23
v5.17.1 1.25
v5.17.2 1.25
v5.17.3 1.25
v5.17.4 1.25
v5.17.5 1.25
v5.17.6 1.25
v5.17.7 1.26
v5.17.8 1.27
v5.17.9 1.27
v5.17.10 1.27
v5.17.11 1.27
v5.18.0 1.27
v5.18.1 1.27
v5.18.2 1.27
v5.18.3 1.27
v5.18.4 1.27
v5.19.0 1.27
v5.19.1 1.27
v5.19.2 1.27
v5.19.3 1.31
v5.19.4 1.32
v5.19.5 1.35
v5.19.6 1.35
v5.19.7 1.35
v5.19.8 1.36
v5.19.9 1.38
v5.19.10 1.38
v5.19.11 1.38
v5.20.0 1.38
v5.20.1 1.38
v5.20.2 1.38
v5.20.3 1.38
v5.21.0 1.38
v5.21.1 1.39
v5.21.2 1.39
v5.21.3 1.39
v5.21.4 1.41
v5.21.5 1.41
v5.21.6 1.41
v5.21.7 1.41
v5.21.8 1.41
v5.21.9 1.41
v5.21.10 1.41
v5.21.11 1.41
v5.22.0 1.41
v5.22.1 1.41
v5.22.2 1.41
v5.22.3 1.41
v5.22.4 1.41
v5.23.0 1.42_01
v5.23.1 1.42_01
v5.23.2 1.42_01
v5.23.3 1.42_01
v5.23.4 1.42_01
v5.23.5 1.42_01
v5.23.6 1.42_01
v5.23.7 1.42_01
v5.23.8 1.42_01
v5.23.9 1.42_02
v5.24.0 1.42_02
v5.24.1 1.42_02
v5.24.2 1.42_02
v5.24.3 1.42_02
v5.24.4 1.42_02
v5.25.0 1.42_02
v5.25.1 1.45_01
v5.25.2 1.45_01
v5.25.3 1.45_01
v5.25.4 1.45_01
v5.25.5 1.45_01
v5.25.6 1.46
v5.25.7 1.46_02
v5.25.8 1.46_02
v5.25.9 1.46_02
v5.25.10 1.46_02
v5.25.11 1.46_02
v5.25.12 1.46_02
v5.26.0 1.46_02
v5.26.1 1.46_02
v5.26.2 1.46_02
v5.26.3 1.46_02
v5.27.0 1.46_02
v5.27.1 1.46_02
v5.27.2 1.48
v5.27.3 1.48
v5.27.4 1.48
v5.27.5 1.49
v5.27.6 1.49
v5.27.7 1.49
v5.27.8 1.49
v5.27.9 1.49
v5.27.10 1.50
v5.27.11 1.50
v5.28.0 1.50
v5.28.1 1.50
v5.28.2 1.50
v5.29.0 1.50
v5.29.1 1.50
v5.29.2 1.50
v5.29.3 1.50
v5.29.4 1.50
v5.29.5 1.50
v5.29.6 1.50
v5.29.7 1.50
v5.29.8 1.50
v5.29.9 1.50
v5.29.10 1.50
v5.30.0 1.50
v5.30.1 1.50
v5.31.0 1.50
v5.31.1 1.50
v5.31.2 1.50
v5.31.3 1.50
v5.31.4 1.52
v5.31.5 1.52
v5.31.6 1.53 Or, more sucinctly:
corelist -a List::Util | perl -ne '/^\s+(\S+)\s+(\S+)/ && $2 >= 1.33 && print && exit'
v5.19.5 1.35 If I find a bug in the module do I have to install a new version of perl to get an update, or can I just download an updated version of the module from the CPAN? And where can I report that bug to?
corelist -u List::Util
Data for 2019-11-20
List::Util was first released with perl v5.7.3
upstream: cpan
bug tracker: https://rt.cpan.org/Public/Dist/Display.html?Name=Scalar-List-Utils(I can get a new version from the CPAN, and that's the bug tracker URL)
corelist -u PerlIO
Data for 2019-11-20
PerlIO was first released with perl v5.7.3
upstream: undef
bug tracker: unknown(PerlIO is shipped with perl, and I report bugs the same way I do with the core language itself, via perlbug)
What do we see if we search for a module that doesn't exist in core (even if we think it should be)
shell$ corelist DateTime
Data for 2019-11-20
DateTime was not in CORE (or so I think)This, of course, doesn't mean that DateTime might not come preinstalled on your system (on macOS for example it's distributed with the operating system.) Just it's not in the main core distribution!
What about something that was added to the core distribution but then removed again?
$ corelist CGI
Data for 2019-11-20
CGI was first released with perl 5.004, deprecated (will be CPAN-only)
in v5.19.7 and removed from v5.21.0What about a core language level feature
shell$ corelist -f say
Data for 2019-11-20
feature "say" was first released with the perl v5.9.5 feature bundle(this means you can use feature "say" on any perl after v5.9.5)
You can even find out what were the module changes between Perl 5.28 and 5.30 for example:
$ corelist --diff v5.28.0 v5.30.0
App::Cpan 1.67 1.672
Archive::Tar 2.30 2.32
Archive::Tar::Constant 2.30 2.32
Archive::Tar::File 2.30 2.32
B 1.74 1.76
B::Concise 1.003 1.004
B::Debug 1.26 (absent)
B::Deparse 1.48 1.49
B::Op_private 5.028000 5.030000
B::Terse 1.08 1.09
CPAN 2.20 2.22
CPAN::Distribution 2.19 2.22
CPAN::Mirrors 2.12 2.21
CPAN::Plugin 0.96 0.97
CPAN::Plugin::Specfile 0.01 0.02
CPAN::Shell 5.5007 5.5008
Compress::Raw::Bzip2 2.074 2.084
Compress::Raw::Zlib 2.076 2.084
Compress::Zlib 2.074 2.084
Config 5.028 5.03
Config::Extensions 0.02 0.03
Config::Perl::V 0.29 0.32
Cwd 3.74 3.78
DB_File 1.840 1.843
Data::Dumper 2.170 2.174
Devel::PPPort 3.40 3.52
Devel::Peek 1.27 1.28
Digest::SHA 6.01 6.02
Encode 2.97 3.01
Encode::Unicode 2.17 2.18
Errno 1.29 1.30
ExtUtils::CBuilder 0.280230 0.280231
ExtUtils::CBuilder::Base 0.280230 0.280231
ExtUtils::CBuilder::Platform::Unix 0.280230 0.280231
ExtUtils::CBuilder::Platform::VMS 0.280230 0.280231
ExtUtils::CBuilder::Platform::Windows 0.280230 0.280231
ExtUtils::CBuilder::Platform::Windows::BCC 0.280230 0.280231
ExtUtils::CBuilder::Platform::Windows::GCC 0.280230 0.280231
ExtUtils::CBuilder::Platform::Windows::MSVC 0.280230 0.280231
ExtUtils::CBuilder::Platform::aix 0.280230 0.280231
ExtUtils::CBuilder::Platform::android 0.280230 0.280231
ExtUtils::CBuilder::Platform::cygwin 0.280230 0.280231
ExtUtils::CBuilder::Platform::darwin 0.280230 0.280231
ExtUtils::CBuilder::Platform::dec_osf 0.280230 0.280231
ExtUtils::CBuilder::Platform::os2 0.280230 0.280231
ExtUtils::Manifest 1.70 1.72
ExtUtils::Miniperl 1.08 1.09
ExtUtils::ParseXS 3.39 3.40
ExtUtils::ParseXS::Constants 3.39 3.40
ExtUtils::ParseXS::CountLines 3.39 3.40
ExtUtils::ParseXS::Eval 3.39 3.40
ExtUtils::ParseXS::Utilities 3.39 3.40
File::Copy 2.33 2.34
File::Find 1.34 1.36
File::Glob 1.31 1.32
File::GlobMapper 1.000 1.001
File::Path 2.15 2.16
File::Spec 3.74 3.78
File::Spec::AmigaOS 3.74 3.78
File::Spec::Cygwin 3.74 3.78
File::Spec::Epoc 3.74 3.78
File::Spec::Functions 3.74 3.78
File::Spec::Mac 3.74 3.78
File::Spec::OS2 3.74 3.78
File::Spec::Unix 3.74 3.78
File::Spec::VMS 3.74 3.78
File::Spec::Win32 3.74 3.78
File::Temp 0.2304 0.2309
Filter::Util::Call 1.58 1.59
GDBM_File 1.17 1.18
HTTP::Tiny 0.070 0.076
I18N::Langinfo 0.17 0.18
IO 1.39 1.40
IO::Compress::Adapter::Bzip2 2.074 2.084
IO::Compress::Adapter::Deflate 2.074 2.084
IO::Compress::Adapter::Identity 2.074 2.084
IO::Compress::Base 2.074 2.084
IO::Compress::Base::Common 2.074 2.084
IO::Compress::Bzip2 2.074 2.084
IO::Compress::Deflate 2.074 2.084
IO::Compress::Gzip 2.074 2.084
IO::Compress::Gzip::Constants 2.074 2.084
IO::Compress::RawDeflate 2.074 2.084
IO::Compress::Zip 2.074 2.084
IO::Compress::Zip::Constants 2.074 2.084
IO::Compress::Zlib::Constants 2.074 2.084
IO::Compress::Zlib::Extra 2.074 2.084
IO::Dir 1.39 1.40
IO::File 1.39 1.40
IO::Handle 1.39 1.40
IO::Pipe 1.39 1.40
IO::Poll 1.39 1.40
IO::Seekable 1.39 1.40
IO::Select 1.39 1.40
IO::Socket 1.39 1.40
IO::Socket::INET 1.39 1.40
IO::Socket::UNIX 1.39 1.40
IO::Uncompress::Adapter::Bunzip2 2.074 2.084
IO::Uncompress::Adapter::Identity 2.074 2.084
IO::Uncompress::Adapter::Inflate 2.074 2.084
IO::Uncompress::AnyInflate 2.074 2.084
IO::Uncompress::AnyUncompress 2.074 2.084
IO::Uncompress::Base 2.074 2.084
IO::Uncompress::Bunzip2 2.074 2.084
IO::Uncompress::Gunzip 2.074 2.084
IO::Uncompress::Inflate 2.074 2.084
IO::Uncompress::RawInflate 2.074 2.084
IO::Uncompress::Unzip 2.074 2.084
IPC::Cmd 1.00 1.02
JSON::PP 2.97001 4.02
JSON::PP::Boolean 2.97001 4.02
Locale::Codes 3.56 (absent)
Locale::Codes::Constants 3.56 (absent)
Locale::Codes::Country 3.56 (absent)
Locale::Codes::Country_Codes 3.56 (absent)
Locale::Codes::Country_Retired 3.56 (absent)
Locale::Codes::Currency 3.56 (absent)
Locale::Codes::Currency_Codes 3.56 (absent)
Locale::Codes::Currency_Retired 3.56 (absent)
Locale::Codes::LangExt 3.56 (absent)
Locale::Codes::LangExt_Codes 3.56 (absent)
Locale::Codes::LangExt_Retired 3.56 (absent)
Locale::Codes::LangFam 3.56 (absent)
Locale::Codes::LangFam_Codes 3.56 (absent)
Locale::Codes::LangFam_Retired 3.56 (absent)
Locale::Codes::LangVar 3.56 (absent)
Locale::Codes::LangVar_Codes 3.56 (absent)
Locale::Codes::LangVar_Retired 3.56 (absent)
Locale::Codes::Language 3.56 (absent)
Locale::Codes::Language_Codes 3.56 (absent)
Locale::Codes::Language_Retired 3.56 (absent)
Locale::Codes::Script 3.56 (absent)
Locale::Codes::Script_Codes 3.56 (absent)
Locale::Codes::Script_Retired 3.56 (absent)
Locale::Country 3.56 (absent)
Locale::Currency 3.56 (absent)
Locale::Language 3.56 (absent)
Locale::Script 3.56 (absent)
Math::BigFloat 1.999811 1.999816
Math::BigFloat::Trace 0.49 0.51
Math::BigInt 1.999811 1.999816
Math::BigInt::Calc 1.999811 1.999816
Math::BigInt::CalcEmu 1.999811 (absent)
Math::BigInt::FastCalc 0.5006 0.5008
Math::BigInt::Lib 1.999811 1.999816
Math::BigInt::Trace 0.49 0.51
Math::BigRat 0.2613 0.2614
Module::CoreList 5.20180622 5.20190522
Module::CoreList::Utils 5.20180622 5.20190522
Module::Load 0.32 0.34
Module::Metadata 1.000033 1.000036
NDBM_File 1.14 1.15
Net::Ping 2.62 2.71
ODBM_File 1.15 1.16
POSIX 1.84 1.88
PerlIO::encoding 0.26 0.27
PerlIO::scalar 0.29 0.30
Pod::Man 4.10 4.11
Pod::ParseLink 4.10 4.11
Pod::Text 4.10 4.11
Pod::Text::Color 4.10 4.11
Pod::Text::Overstrike 4.10 4.11
Pod::Text::Termcap 4.10 4.11
SDBM_File 1.14 1.15
Storable 3.08 3.15
Storable::Limit (undef) (absent)
Test2 1.302133 1.302162
Test2::API 1.302133 1.302162
Test2::API::Breakage 1.302133 1.302162
Test2::API::Context 1.302133 1.302162
Test2::API::Instance 1.302133 1.302162
Test2::API::Stack 1.302133 1.302162
Test2::Event 1.302133 1.302162
Test2::Event::Bail 1.302133 1.302162
Test2::Event::Diag 1.302133 1.302162
Test2::Event::Encoding 1.302133 1.302162
Test2::Event::Exception 1.302133 1.302162
Test2::Event::Fail 1.302133 1.302162
Test2::Event::Generic 1.302133 1.302162
Test2::Event::Note 1.302133 1.302162
Test2::Event::Ok 1.302133 1.302162
Test2::Event::Pass 1.302133 1.302162
Test2::Event::Plan 1.302133 1.302162
Test2::Event::Skip 1.302133 1.302162
Test2::Event::Subtest 1.302133 1.302162
Test2::Event::TAP::Version 1.302133 1.302162
Test2::Event::V2 1.302133 1.302162
Test2::Event::Waiting 1.302133 1.302162
Test2::EventFacet 1.302133 1.302162
Test2::EventFacet::About 1.302133 1.302162
Test2::EventFacet::Amnesty 1.302133 1.302162
Test2::EventFacet::Assert 1.302133 1.302162
Test2::EventFacet::Control 1.302133 1.302162
Test2::EventFacet::Error 1.302133 1.302162
Test2::EventFacet::Hub 1.302133 1.302162
Test2::EventFacet::Info 1.302133 1.302162
Test2::EventFacet::Info::Table (absent) (undef)
Test2::EventFacet::Meta 1.302133 1.302162
Test2::EventFacet::Parent 1.302133 1.302162
Test2::EventFacet::Plan 1.302133 1.302162
Test2::EventFacet::Render 1.302133 1.302162
Test2::EventFacet::Trace 1.302133 1.302162
Test2::Formatter 1.302133 1.302162
Test2::Formatter::TAP 1.302133 1.302162
Test2::Hub 1.302133 1.302162
Test2::Hub::Interceptor 1.302133 1.302162
Test2::Hub::Interceptor::Terminator 1.302133 1.302162
Test2::Hub::Subtest 1.302133 1.302162
Test2::IPC 1.302133 1.302162
Test2::IPC::Driver 1.302133 1.302162
Test2::IPC::Driver::Files 1.302133 1.302162
Test2::Tools::Tiny 1.302133 1.302162
Test2::Util 1.302133 1.302162
Test2::Util::ExternalMeta 1.302133 1.302162
Test2::Util::Facets2Legacy 1.302133 1.302162
Test2::Util::HashBase 1.302133 1.302162
Test2::Util::Trace 1.302133 1.302162
Test::Builder 1.302133 1.302162
Test::Builder::Formatter 1.302133 1.302162
Test::Builder::Module 1.302133 1.302162
Test::Builder::Tester 1.302133 1.302162
Test::Builder::Tester::Color 1.302133 1.302162
Test::Builder::TodoDiag 1.302133 1.302162
Test::More 1.302133 1.302162
Test::Simple 1.302133 1.302162
Test::Tester 1.302133 1.302162
Test::Tester::Capture 1.302133 1.302162
Test::Tester::CaptureRunner 1.302133 1.302162
Test::Tester::Delegate 1.302133 1.302162
Test::use::ok 1.302133 1.302162
Thread::Queue 3.12 3.13
Time::HiRes 1.9759 1.9760
Time::Local 1.25 1.28
Time::Piece 1.3204 1.33
Time::Seconds 1.3204 1.33
Unicode 10.0.0 12.1.0
Unicode::Collate 1.25 1.27
Unicode::Collate::CJK::Big5 1.25 1.27
Unicode::Collate::CJK::GB2312 1.25 1.27
Unicode::Collate::CJK::JISX0208 1.25 1.27
Unicode::Collate::CJK::Korean 1.25 1.27
Unicode::Collate::CJK::Pinyin 1.25 1.27
Unicode::Collate::CJK::Stroke 1.25 1.27
Unicode::Collate::CJK::Zhuyin 1.25 1.27
Unicode::Collate::Locale 1.25 1.27
Unicode::UCD 0.70 0.72
User::grent 1.02 1.03
XS::APItest 0.98 1.00
XS::Typemap 0.16 0.17
arybase 0.15 (absent)
bigint 0.49 0.51
bignum 0.49 0.51
bigrat 0.49 0.51
bytes 1.06 1.07
deprecate 0.03 0.04
experimental 0.019 0.020
feature 1.52 1.54
lib 0.64 0.65
ok 1.302133 1.302162
parent 0.236 0.237
perlfaq 5.021011 5.20190126
re 0.36 0.37
sigtrap 1.08 1.09
threads::shared 1.58 1.60
utf8 1.21 1.22
vars 1.04 1.05
version 0.9923 0.9924
version::regex 0.9923 0.9924
warnings 1.42 1.44Beyond a Core Workout
One thing to consider however is that core modules aren't your own source of modules on the enviroment.
Consider running a Lambda function and using Try::Tiny. Not a core module:
$ corelist Try::Tiny
Data for 2019-11-20
Try::Tiny was not in CORE (or so I think)But you can still use it in an AWS::Lambda powered function without having to supply it yourself in the zipfile! How is this possible? Well, it's because AWS::Lambda itself uses the module, and therefore it's in the default layer that your Lambda function uses.
What you need to do is look at the dependance graph for AWS::Lambda so you can see not only the direct dependencies of AWS::Lambda but its dependencies' dependencies and so on. You can probably use all of these too!
You can literally see a graph for each module's dependancies by visiting https://cpandeps.grinnz.com/. For example, here's the graph for AWS::Lambda.
Similarly, we noted that DateTime is available on macOS Catalina. Therefore, any runtime dependencies it requires must also be installed, correct? Well, it's not quite that easy. You'd expect the excellent Params::ValidationCompiler to be available as it's a requirement of the DateTime 1.51 module that's on the CPAN. But macOS only bundles version 1.06, which doesn't require Params::ValidationCompiler, so it's not there. You have to sweat the details!
Keep Working It
Of course, if you need something from the CPAN, you should go ahead and grab it. But, sometimes it's good to run slim...
This article contributed by: Mark Fowler <mark@twoshortplanks.com>