Perl Advent Calendar 2018http://perladvent.org/2018/2024-03-13T21:03:14ZMark FowlerXML::Atom::SimpleFeedMerry Christmashttp://perladvent.org/2018/2018-12-25.html<div class='pod'><p>So here we are, Merry Christmas!<br> Everybody's been having fun!<br> Look to the future now<br> It's only just begun
</p>
<p>So concludes the 19th year of the Perl Advent Calendar. We're already looking forward to the big two zero next year (and if you're inspired by this year it's never to early to <a href="http://cfp.perladvent.org/">sign up as an author</a> for next year's calendar.) In the meantime I owe a tremendous debt of gratitude to everyone who's written an article (some people two or three), helped with the hosting, and to those that sent us pull requests for my many many typos or provided improvements and clarifications on articles. So, without further ado let me thank:</p>
<p><li>Andrew Solomon</li> <li>Barbie</li> <li>Branislav Zahradník</li> <li>Dave Cross</li> <li>E. Choroba</li> <li>José Joaquín Atria</li> <li>Joel Berger</li> <li>Mohammad S Anwar</li> <li>Nick Tonkin</li> <li>Pete Houston</li> <li>Ricardo Signes</li> <li>Tony Edwardson</li>
</p>
<p>I literally couldn't have done it without you people, so thank you from the bottom of my heart.</p>
<p>Of course, as has been the case for many years now, this isn't the only technology advent calendar. There's several more calendars about Perl that if you haven't been following I urge you to head off and enjoy:</p>
<p><ul> <li><a href="https://perl6advent.wordpress.com/">Perl 6 Advent Calendar</a></li> <li><a href="https://perl6.online/">Perl 6 one-liners Advent Calendar</a></li> <li><a href="https://mojolicious.io/">Mojolicious Advent Calendar</a></li> <li><a href="http://advent.perldancer.org/2018">Perl Dancer Advent Calendar</a></li> </ul>
</p>
<p>However, you shouldn't stop with just Perl. These days there's a plethora of technology advent calendars that's worth reading and are aggregated at <a href="http://www.lenjaffe.com/AdventPlanet/">Advent Planet</a></p>
<p>Finally, if you love Perl, and the fact that you've been reading these calendars indicates you do, consider supporting the ongoing development of Perl with a <a href="https://perlfoundation.z2systems.com/np/clients/perlfoundation/donation.jsp">donation</a> to the Perl Foundation. You could even set up a reoccurring payment as an early New Year's resolution.</p>
<p>Merry Christmas from me to yours, and a Happy New Year.</p>
<p>Mark.</p>
</div>2018-12-25T00:00:00ZMark FowlerWatching The Perl Conference Againhttp://perladvent.org/2018/2018-12-24.html<div class='pod'><p>Another year, another successful Christmas Eve. It was time for the Wise Old Elf to pack his bags and head off for his southern-hemisphere summer holiday once again.</p>
<p>This year, <a href="http://www.perladvent.org/2017/2017-12-24.html">like last year</a>, before he left he decided to download a bunch of videos to watch on the beach. What better to entertain himself than the talks from last year's Perl Conference?</p>
<p><center><iframe width="560" height="315" src="https://www.youtube.com/embed/videoseries?list=PLA9_Hq3zhoFx1inHPmwmbXrPcI8cc0fyO&hl=en_US" frameborder="0" allowfullscreen></iframe></center>
</p>
<p>Downloading these with Perl was just as easy as last year:</p>
<pre><code> perl -Mojo -E \
'g(shift)->dom(".pl-video-title-link")->map(sub{say $_->attr("href")})' \
'https://www.youtube.com/playlist?list=PLA9_Hq3zhoFx1inHPmwmbXrPcI8cc0fyO' \
| xargs -n 1 youtube-download</code></pre>
<h3 id="Sharing-is-Caring">Sharing is Caring</h3>
<p>On the beach, Mrs Wiser Elf looked over the Wise Old Elf's shoulder.</p>
<p>"Watcha watching?", she asked.</p>
<p>"The videos from the The Perl Conference in Salt Lake City this year. I'm watching this <a href="https://www.youtube.com/watch?v=tdh0w69h5xI&index=28&list=PLA9_Hq3zhoFx1inHPmwmbXrPcI8cc0fyO&t=0s">talk</a> on the relaunching of <a href="http://www.perl.com">perl.com</a>."</p>
<p>"Oh! Interesting" she said. Mrs Wiser Elf had been one of the best Perl programmers Santa had had before she'd moved over to designing the custom chipsets that Santa uses in his datacenters.</p>
<p>"I'd love to watch a few of those on my tablet", she continued, "but the bandwidth here is <i>terrible</i>."</p>
<p>"Don't worry my sweet, you can play them right off of my laptop."</p>
<p>The Wise Old Elf had installed <a href="https://metacpan.org/module/App::HTTPThis">App::HTTPThis</a> which would allow him to quickly serve any directory as a website. In the shell The Wise Old Elf changed directory to where he'd downloaded the videos and typed one command:</p>
<pre><code> bash$ http_this
Exporting '.', available at:
http://127.0.0.1:7007/</code></pre>
<p>The Wise Old Elf clicked the link and was greeted by a listing of all the files in his browser.</p>
<center><img src="listing.jpg" width="600" height="435" alt="listing"></center>
<p>"Okay my gumdrop, you can head over to http://wiseoldlaptop.local port 7007 to see all the videos"</p>
<p>"I shall do no such thing! What have I always told you about using https these days. Anyone could be snooping on what we're doing."</p>
<p>"Sorry my love".</p>
<p>The Wise Old Elf quickly typed hit ctrl-c to kill the webserver and typed a different command:</p>
<pre><code> bash$ https_this
Generating RSA private key, 2048 bit long modulus
.....+++
.....................................................................+++
e is 65537 (0x10001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:
State or Province Name (full name) []:
Locality Name (eg, city) []:
Organization Name (eg, company) []:
Organizational Unit Name (eg, section) []:
Common Name (eg, fully qualified host name) []:localhost
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
Signature ok
subject=/CN=localhost
Getting Private key
2018/12/03-06:11:12 Starman::Server (type Net::Server::PreFork) starting! pid(1130)
Resolved [*]:8443 to [0.0.0.0]:8443, IPv4
Binding to SSL port 8443 on host 0.0.0.0 with IPv4
Setting gid to "20 20 20 12 61 79 80 81 98 701 33 100 204 250 395 398 399"
Exporting '.', available at:
https://*:8443/</code></pre>
<p>The <a href="https://metacpan.org/module/App::HTTPSThis">App::HTTPSThis</a> module had just automatically created a <i>snake oil</i> certificate, meaning that the connection was encrypted, but still vunerable to man-in-the-middle attacks when the connection was first established - not that that mattered in this case.</p>
<p>"Okay my pecan pie, you can go to <b>https</b>://wiseoldlaptop.local port 8443 now"</p>
<p>And with that, the two elves could get back to taking a well earned rest...until it was time to get ready for next year.</p>
</div>2018-12-24T00:00:00ZMark FowlerLegacy code strikes againhttp://perladvent.org/2018/2018-12-23.html<div class='pod'><p>It's 3 o'clock in the morning. Our poor programmer is finally asleep. It's the run up to Christmas and work has been hectic with the increased website traffic. But all is good. All is well. And finally the programmer can dream of a life where...</p>
<p><i>ding ding, ding ding</i></p>
<p>"Our databases are stuck AGAIN, we are NOT delivering, do something! NOW!"</p>
<p>Bleary eyed our protagonist rolls out of bed and logs into the production systems. Our programmer-soon-hero-again has discovered that database servers are overloaded with unused connections.</p>
<h3 id="Code-architecture">Code architecture</h3>
<p>In order to process every gift delivery the code has to access several databases:</p>
<p><ul> <li>The delivery queue database</li> <li>The accounting department database</li> <li>The legal department database</li> <li>The compliance database</li> </ul>
</p>
<p>The location of every database for every gift's delivery region is located in the central key-value storage, which at least is holding up under the increased website traffic.</p>
<p>The problem is that gift processing opens all four connections when it first it starts up keeping the connections open idle between gift processing. With the increased workload during the run up to Christmas there's too many processes with open database connections!</p>
<h3 id="Refactoring">Refactoring</h3>
<p>If our programmer is ever going to get a full night's sleep, he has to refactor the code quickly.</p>
<p>The solution is "easy". Do lazy connections! While the code is off doing other things that don't involve accessing the database there's no reason it should have the connections open.</p>
<h4 id="Naive-solution">Naive solution</h4>
<p>The naive solution would be to connect only when needed - making a new connection in each function that needs to access the database.</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">foo</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$delivery_queue_db</span> <span class="operator">=</span> <span class="word">delivery_queue_connect</span><span class="structure">(</span> <span class="magic">@_</span> <span class="structure">);</span><br /> <span class="operator">...</span><br /><span class="structure">}</span><br /><br /><span class="keyword">sub</span> <span class="word">bar</span> <span class="structure">{</span><br /> <span class="keyword">if</span> <span class="structure">(</span><span class="keyword">my</span> <span class="symbol">$delivery_queue_db</span> <span class="operator">=</span> <span class="word">delivery_queue_connect</span><span class="structure">(</span> <span class="magic">@_</span> <span class="structure">))</span> <span class="structure">{</span><br /> <span class="operator">...</span><br /> <span class="structure">}</span><br /><span class="structure">}</span></code></pre>
<p>Unfortunately that makes the performance even worse! The same connection is required by many independent parts of the code and reconnecting in every single function overloads the database server even more!</p>
<h4 id="OOP-refactoring">OOP refactoring</h4>
<p>In an ideal world our programmer would refactor the code to work in an object orientated fashion so that each database connection is available from an object accessor only where it was needed.</p>
<p>Unfortunately that is going to be a <b>huge</b> amount of work. The programmer did a bit of estimating, and figured out he could get it all done by the end of March...and he's like to sleep sometime before then.</p>
<h4 id="Legacy-solution">Legacy solution</h4>
<p>One solution is to make use of global variables, and <i>localize</i> them so they're automatically cleared up as execution leaves scope (and the database connections will automatically be dropped)</p>
<p>For example.</p>
<pre><code class="code-listing"><span class="comment"># our global package variables<br /></span><span class="keyword">our</span> <span class="structure">(</span><span class="symbol">$delivery_queue_db</span><span class="operator">,</span> <span class="symbol">$accounting_db</span><span class="operator">,</span> <span class="operator">...</span><span class="structure">);</span><br /><br /><span class="comment"># do the bit of code that needs the same database connections<br /></span><span class="keyword">sub</span> <span class="word">handle_gift</span> <span class="structure">{</span><br /><span class="comment"> # localize the connections so when we leave scope they'll<br /> # be automatically cleared up<br /></span> <span class="keyword">local</span> <span class="structure">(</span><span class="symbol">$delivery_queue_db</span><span class="operator">,</span> <span class="symbol">$accounting_db</span><span class="operator">,</span> <span class="operator">...</span><span class="structure">);</span><br /><br /><span class="comment"> # use functions that connect to the various databases<br /> # the first time a connection to the database is needed<br /></span> <span class="word">record_shipping_timeline</span><span class="structure">(</span><span class="operator">...</span><span class="structure">);</span><br /> <span class="word">update_stock_count</span><span class="structure">(</span><span class="operator">...</span><span class="structure">);</span><br /> <span class="operator">...</span><br /><span class="structure">}</span><br /><br /><span class="comment"># some code that is eventually called by something<br /># record_shipping_timeline eventually calls<br /></span><span class="keyword">sub</span> <span class="word">calculate_shipping_method</span> <span class="structure">{</span><br /><span class="comment"> # in each function that needs a connection to the<br /> # database we connect if we need to if the global variable<br /> # doesn't already have a connection<br /></span> <span class="symbol">$delivery_queue_db</span> <span class="operator">//=</span> <span class="word">delivery_queue_connect</span><span class="structure">(</span> <span class="magic">@_</span> <span class="structure">);</span><br /><br /> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<p>It's a lot better. Now the database connections aren't created until the very point they're about to be used. But it's still a lot of additional code to maintain, and the programmer will have well over 100 such parameters.</p>
<p>And now he will also have to extend every function with connectivity information, or make that yet another global variable too.</p>
<p>Trembling between 100 variables, 10_000 functions and who knows how many lines of code our programmer suddenly hears something start to whisper. Has the sleep deprivation finally sent him over the edge?</p>
<p>... "Use me, search me, use me, search me"</p>
<p>"Is it you, Almighty CPAN? Do you have solution for me?"</p>
<p>"Take a look at Context::Singleton"</p>
<h4 id="Curiosity-driven-development-solution">Curiosity driven development solution</h4>
<p><a href="https://metacpan.org/module/Context::Singleton">Context::Singleton</a> is a way of handling shared state in a much more controlled and organized fashion. It handles the localization and allows a centralized place for declaring a <i>recipe</i> for deducing shared state when it's first needed.</p>
<pre><code class="code-listing"><span class="comment"># define a recipe for deducing the delivery_queue_db<br /># whenever we need it<br /></span><span class="word">contrive</span> <span class="single">'delivery_queue_db'</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">dep</span> <span class="operator">=></span> <span class="structure">[</span><span class="words">qw[ gift ... ]</span><span class="structure">]</span><span class="operator">,</span><br /> <span class="word">as</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$gift</span><span class="operator">,</span> <span class="operator">...</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span> <span class="operator">...</span> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="keyword">sub</span> <span class="word">foo</span> <span class="structure">{</span><br /><span class="comment"> # get the delivery_queue_db for the current frame<br /> # calling the logic in contrive to create a new<br /> # connection if that hasn't already happened this frame<br /></span> <span class="keyword">my</span> <span class="symbol">$delivery_queue_db</span> <span class="operator">=</span> <span class="word">deduce</span> <span class="single">'delivery_queue_db'</span><span class="structure">;</span><br /> <span class="operator">...</span><br /><span class="structure">}</span><br /><br /><span class="keyword">sub</span> <span class="word">bar</span> <span class="structure">{</span><br /><span class="comment"> # this is the same thing as in foo above - getting<br /> # the delivery_queue_db for the current frame and<br /> # using the logic in contrive to do create a new<br /> # connection if delivery_queue_db doesn't exist in<br /> # the current frame, but here we're also using<br /> # try_deduce to see if we have all the dependencies<br /> # to do that first<br /></span> <span class="keyword">if</span> <span class="structure">(</span><span class="word">try_deduce</span> <span class="single">'delivery_queue_db'</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$delivery_queue_db</span> <span class="operator">=</span> <span class="word">deduce</span> <span class="single">'delivery_queue_db'</span><span class="structure">;</span><br /> <span class="operator">...</span><br /> <span class="structure">}</span><br /><span class="structure">}</span><br /><br /><span class="keyword">sub</span> <span class="word">handle_gift</span> <span class="structure">{</span><br /><span class="comment"> # start a new context. Things that we deduce will be<br /> # cleared up when execution leaves the frame<br /></span> <span class="word">frame</span> <span class="structure">{</span><br /><span class="comment"> # declare the gift in our current frame. Now<br /> # we don't have to pass it around to the functions<br /> # that we call from within the frame since they<br /> # can access it via the context singleton keywords<br /></span> <span class="word">proclaim</span> <span class="single">'gift'</span> <span class="operator">=></span> <span class="symbol">$gift</span><span class="structure">;</span><br /><br /> <span class="operator">...</span><span class="structure">;</span><br /> <span class="structure">}</span><br /><span class="structure">}</span></code></pre>
<p>By using Context::Singleton our programmer has removed four arguments from every important function - resulting largest code reducing commit in the project history!</p>
<p>Each connection is now created first time it is needed and persisted till end of the frame. The programmer's system finally works again, and gifts are being delivered.</p>
<p>This approach even allowed a long desired optimization (previously not implemented due to "argument hell") so now it's most likely system will handle even next Christmas' load ...</p>
<h3 id="How-it-works">How it works</h3>
<p><a href="https://metacpan.org/module/Context::Singleton">Context::Singleton</a> offers data control mechanism similar to <code>my</code> variable within code blocks, but independent of code architecture and execution flow. Things within the frame will always exist just until the end of the frame so will be cleared up automatically avoiding globals existing throughout the entire codebase.</p>
<p>It affords reasonable caching as well - derived properties are stored in same <code>frame</code> as their requirements. In our case, <code>delivery_queue_db</code> will always be populated in same frame as <code>gift</code> value was set.</p>
<p>It provides multiple ways how to populate variables based on availability of other variables. The simple <code>contrive</code> example above is just one simple implementation - Context::Singleton also allows you to specify recipies where you can specify class names, builder methods to be called, or defaults and values.</p>
<p>It handles a lot of the complex stuff hand rolling a simple solution with a combination of the <code>local</code> keyword and accessor subroutines can't handle alone: It gives immutability on scope, and can build values based on dependencies. With dependency tracking it can rebuild them in inner scope when their dependencies have been modified! And it is able to handle cyclic dependencies as well.</p>
<p>One way to think about it is that it acts like <code>$_</code> providing "default object" where every deducible property acts like method (with some kind of multi-dispatch) though you can compose specific classes as you need using / providing different <code>contrive</code> recipes.</p>
<p>See <a href="https://www.youtube.com/watch?v=KBQ0prTu5ys">TPC in Amsterdam presentation</a> (<a href="https://rawgit.com/happy-barney/slides/master/2017-08-09%20-%20TPCiA%20-%20Context%20Singleton/src/tpcia-2017-08-09-context-singleton.html">slides</a>) for more.</p>
</div>2018-12-23T00:00:00ZBranislav ZahradníkValidationhttp://perladvent.org/2018/2018-12-22.html<div class='pod'><p>Today I want to talk about <a href="https://metacpan.org/module/Method::ParamValidator">Method::ParamValidator</a> - yet another validator for method parameters.</p>
<p>Why another validator? I was looking for param validators that can be shared among various Google API interfaces that I have worked on. None of the existing solutions worked for me. So I decided to create simple, yet easily configurable, param validator. The best part of this solution is that you can add the param validation programmatically as well as in a configuration file.</p>
<p>I think the Christmas Elves would find this flexibility really useful when working on Santa's codebase.</p>
<h3 id="Configuring-Validation">Configuring Validation</h3>
<p>There are two core methods <code>add_field()</code> and <code>add_method()</code> that can be used to setup a LMethod::ParamValidator validator.</p>
<h4 id="add_field-params">add_field(\%params)</h4>
<p>First step in creating a validator is to add the fields that can be shared by one or many methods in a validator. You can provide the following keys to add new field.</p>
<p><table border=1 frame=void rules=rows> <tr> <td><b>name</b></td> <td>unique field name (required).</td> </tr> <tr> <td><b>format</b></td> <td>data type of the field (optional), possible values are 's' (for string) and 'd' (for digits). Default is 's'.</td> </tr> <tr> <td><b>check</b></td> <td>code ref for custom check of the field (optional).</td> </tr> <tr> <td><b>source</b></td> <td>lookup hashref for the acceptable values for the field (optional).</td> </tr> <tr> <td><b>message</b></td> <td>test message (optional).</td> </tr> </table>
</p>
<h4 id="add_method-params">add_method(\%params)</h4>
<p>After adding fields to the validator, it is time to add method to be validated. You can setup method by providing the following keys.</p>
<p><table border=1 frame=void rules=rows> <tr><td><b>name</b></td><td>unique method name (required).</td></td> <tr><td><b>fields<b></td><td>hashref with field names.</td></tr> </table>
</p>
<h3 id="Example">Example</h3>
<p>Suppose our elves want to validate the parameters for a method called <code>add_child()</code>. It requires parameters passed as a hashref with required keys <code>firstname</code>, <code>lastname</code> and <code>age</code> and it also accepts an optional key <code>notes</code> as well. We will first add all the fields first and then we will add the method the method that uses these fields.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Method::ParamValidator</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$validator</span> <span class="operator">=</span> <span class="word">Method::ParamValidator</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /><span class="comment"># Add fields<br /></span><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_field</span><span class="structure">({</span> <span class="word">name</span> <span class="operator">=></span> <span class="single">'firstname'</span><span class="operator">,</span> <span class="word">format</span> <span class="operator">=></span> <span class="single">'s'</span> <span class="structure">});</span><br /><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_field</span><span class="structure">({</span> <span class="word">name</span> <span class="operator">=></span> <span class="single">'lastname'</span><span class="operator">,</span> <span class="word">format</span> <span class="operator">=></span> <span class="single">'s'</span> <span class="structure">});</span><br /><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_field</span><span class="structure">({</span> <span class="word">name</span> <span class="operator">=></span> <span class="single">'age'</span><span class="operator">,</span> <span class="word">format</span> <span class="operator">=></span> <span class="single">'d'</span> <span class="structure">});</span><br /><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_field</span><span class="structure">({</span> <span class="word">name</span> <span class="operator">=></span> <span class="single">'notes'</span><span class="operator">,</span> <span class="word">format</span> <span class="operator">=></span> <span class="single">'s'</span> <span class="structure">});</span><br /><br /><span class="comment"># Add method<br /></span><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_method</span><span class="structure">({</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'add_child'</span><span class="operator">,</span><br /> <span class="word">fields</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="word">firstname</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="word">lastname</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="word">notes</span> <span class="operator">=></span> <span class="number">0</span><br /> <span class="structure">}</span><br /><span class="structure">});</span></code></pre>
<p>We can alternatively setup the validator using a configuration file. If we wanted the same checks as above we could create a JSON file like so:</p>
<pre><code class="code-listing"><span class="synIdentifier">{</span> <span class="synConstant">"fields"</span> : <span class="synIdentifier">[</span> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"firstname"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"s"</span> <span class="synIdentifier">}</span>,<br /> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"lastname"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"s"</span> <span class="synIdentifier">}</span>,<br /> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"age"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"d"</span> <span class="synIdentifier">}</span>,<br /> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"notes"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"s"</span> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">]</span>,<br /> <span class="synConstant">"methods"</span> : <span class="synIdentifier">[</span> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"add_child"</span>,<br /> <span class="synConstant">"fields"</span>: <span class="synIdentifier">{</span> <span class="synConstant">"firstname"</span> : <span class="synConstant">"1"</span>,<br /> <span class="synConstant">"lastname"</span> : <span class="synConstant">"1"</span>,<br /> <span class="synConstant">"age"</span> : <span class="synConstant">"1"</span>,<br /> <span class="synConstant">"notes"</span> : <span class="synConstant">"0"</span><br /> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">]</span><br /><span class="synIdentifier">}</span></code></pre>
<p>And now when we instanciate our Method::ParamValidator instance we can simply pass in the name of the configuraton file <code>config.json</code>.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Method::ParamValidator</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$validator</span> <span class="operator">=</span> <span class="word">Method::ParamValidator</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span> <span class="word">config</span> <span class="operator">=></span> <span class="double">"config.json"</span> <span class="structure">});</span></code></pre>
<h3 id="Using-Our-Validator">Using Our Validator</h3>
<p>Our validator is very simple to use; A call to <code>validate</code> passing in the method name and the parameters will simply throw an exception if the validation fails. We can either let these exceptions terminate our program, or we can catch them using Perl's exception handling (for example with <a href="https://metacpan.org/module/Try::Tiny">Try::Tiny</a>'s <code>try</code>/<code>catch</code> blocks or even with the inbuilt <code>eval</code> keyword.)</p>
<p>Let's demonstrate what kind of error messages the elves are going to get with a test suite. <a href="https://metacpan.org/module/Test::Exception">Test::Exception</a> expects an exception to be thrown inside a <code>throws_ok</code> block and conversely fails if one is thrown inside a <code>lives_ok</code> block.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Test::More</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Exception</span><span class="structure">;</span><br /><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'get_xyz'</span><span class="structure">)</span> <span class="structure">}</span> <span class="regexp">qr/Invalid method name received/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="structure">)</span> <span class="structure">}</span> <span class="regexp">qr/Missing parameters/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">[])</span> <span class="structure">}</span> <span class="regexp">qr/Invalid parameters data structure/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="single">'A'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="number">10</span><span class="operator">,</span> <span class="word">notes</span> <span class="operator">=></span> <span class="single">'s'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Missing required parameter/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="number">10</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Undefined required parameter/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Missing required parameter/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="number">40</span><span class="operator">,</span> <span class="word">location</span> <span class="operator">=></span> <span class="single">'X'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint/</span><span class="structure">;</span><br /><span class="word">lives_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="number">40</span><span class="operator">,</span> <span class="word">location</span> <span class="operator">=></span> <span class="single">'UK'</span> <span class="structure">})</span> <span class="structure">};</span><br /><span class="word">lives_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'add_child'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">firstname</span> <span class="operator">=></span> <span class="single">'F'</span><span class="operator">,</span> <span class="word">lastname</span> <span class="operator">=></span> <span class="single">'L'</span><span class="operator">,</span> <span class="word">age</span> <span class="operator">=></span> <span class="number">40</span><span class="operator">,</span> <span class="word">location</span> <span class="operator">=></span> <span class="single">'uk'</span> <span class="structure">})</span> <span class="structure">};</span><br /><br /><span class="word">done_testing</span><span class="structure">();</span></code></pre>
<h3 id="Custom-Checks">Custom Checks</h3>
<p>Up until this point we've used very simple inbuilt checks: Is this a string? Is this digits? Are the required parameters passed? But what if we want to define something more complicated?</p>
<p>When adding field to a validator, you can hookup your own checks. Below we are adding new field <code>location</code> with a custom check</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$LOCATION</span> <span class="operator">=</span> <span class="structure">{</span> <span class="single">'USA'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span> <span class="single">'UK'</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">};</span><br /><br /><span class="comment"># Add field with custom check<br /></span><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_field</span><span class="structure">({</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'location'</span><span class="operator">,</span><br /> <span class="word">format</span> <span class="operator">=></span> <span class="single">'s'</span><span class="operator">,</span><br /> <span class="word">check</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="word">exists</span> <span class="symbol">$LOCATION</span><span class="operator">-></span><span class="structure">{</span> <span class="word">uc</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">[</span><span class="number">0</span><span class="structure">])</span> <span class="structure">}</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="comment"># Add method using the new field with custom check<br /></span><span class="symbol">$validator</span><span class="operator">-></span><span class="word">add_method</span><span class="structure">({</span> <span class="word">name</span> <span class="operator">=></span> <span class="single">'check_location'</span><span class="operator">,</span> <span class="word">fields</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">location</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">}});</span></code></pre>
<p>While we can't create custom Perl code in our configuration file we can create a limited custom check as above:</p>
<pre><code class="code-listing"><span class="synIdentifier">{</span><br /> <span class="synConstant">"fields"</span> : <span class="synIdentifier">[{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"location"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"s"</span>, <span class="synConstant">"source"</span>: <span class="synIdentifier">[</span> <span class="synConstant">"USA"</span>, <span class="synConstant">"UK"</span> <span class="synIdentifier">]</span> <span class="synIdentifier">}</span> <span class="synIdentifier">]</span>,<br /> <span class="synConstant">"methods"</span> : <span class="synIdentifier">[{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"check_location"</span>, <span class="synConstant">"fields"</span>: <span class="synIdentifier">{</span> <span class="synConstant">"location"</span> : <span class="synConstant">"1"</span> <span class="synIdentifier">}</span> <span class="synIdentifier">}</span> <span class="synIdentifier">]</span><br /><span class="synIdentifier">}</span></code></pre>
<p>The array under the <code>source</code> key lists all the values that the <code>location</code> field can have whenever the value is uppercased.</p>
<p>We can demonstrate what kind of error messages our elves are going to see with another test script:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Test::More</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Exception</span><span class="structure">;</span><br /><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="single">'check_location'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">location</span> <span class="operator">=></span> <span class="single">'X'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint/</span><span class="structure">;</span><br /><span class="word">done_testing</span><span class="structure">();</span></code></pre>
<h3 id="Extend-Moo-Package">Extend Moo Package</h3>
<p>So we've seen how we can call the validation up manually, but is there an easy way to add it to multiple methods in a class without having to change the method code?</p>
<p>If you want to plug the validator into an existing <code>Moo</code> package it's easy. For an example let's create package <code>Calculator</code>.</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">Calculator</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Moo</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">calc</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$self</span><span class="operator">,</span> <span class="symbol">$param</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="keyword">if</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">op</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'add'</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">return</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">a</span><span class="structure">}</span> <span class="operator">+</span> <span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">b</span><span class="structure">});</span><br /> <span class="structure">}</span><br /> <span class="keyword">elsif</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">op</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'sub'</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">return</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">a</span><span class="structure">}</span> <span class="operator">-</span> <span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">b</span><span class="structure">});</span><br /> <span class="structure">}</span><br /> <span class="keyword">elsif</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">op</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'mul'</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">return</span> <span class="structure">(</span><span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">a</span><span class="structure">}</span> <span class="operator">*</span> <span class="symbol">$param</span><span class="operator">-></span><span class="structure">{</span><span class="word">b</span><span class="structure">});</span><br /> <span class="structure">}</span><br /><span class="structure">}</span></code></pre>
<p>Now it is time to create configuration file <code>calc.json</code> for validator.</p>
<pre><code class="code-listing"><span class="synIdentifier">{</span> <span class="synConstant">"fields"</span> : <span class="synIdentifier">[</span> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"op"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"s"</span>, <span class="synConstant">"source"</span>: <span class="synIdentifier">[</span> <span class="synConstant">"add"</span>, <span class="synConstant">"sub"</span>, <span class="synConstant">"mul"</span> <span class="synIdentifier">]</span> <span class="synIdentifier">}</span>,<br /> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"a"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"d"</span> <span class="synIdentifier">}</span>,<br /> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"b"</span>, <span class="synConstant">"format"</span> : <span class="synConstant">"d"</span> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">]</span>,<br /> <span class="synConstant">"methods"</span> : <span class="synIdentifier">[</span> <span class="synIdentifier">{</span> <span class="synConstant">"name"</span> : <span class="synConstant">"calc"</span>,<br /> <span class="synConstant">"fields"</span>: <span class="synIdentifier">{</span> <span class="synConstant">"op"</span> : <span class="synConstant">"1"</span>,<br /> <span class="synConstant">"a"</span> : <span class="synConstant">"1"</span>,<br /> <span class="synConstant">"b"</span> : <span class="synConstant">"1"</span><br /> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">}</span><br /> <span class="synIdentifier">]</span><br /><span class="synIdentifier">}</span></code></pre>
<p>Add the following lines to plug the validator.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Method::ParamValidator</span><span class="structure">;</span><br /><br /><span class="word">has</span> <span class="single">'validator'</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">default</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span> <span class="word">Method::ParamValidator</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><span class="word">config</span> <span class="operator">=></span> <span class="double">"calc.json"</span><span class="structure">)</span> <span class="structure">}</span><br /><span class="structure">);</span><br /><br /><span class="keyword">foreach</span> <span class="keyword">my</span> <span class="symbol">$method</span> <span class="structure">(</span><span class="words">qw/calc/</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="word">before</span> <span class="symbol">$method</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$self</span><span class="operator">,</span> <span class="symbol">$param</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">validator</span><span class="operator">-></span><span class="word">validate</span><span class="structure">(</span><span class="symbol">$method</span><span class="operator">,</span> <span class="symbol">$param</span><span class="structure">);</span><br /> <span class="structure">};</span><br /><span class="structure">}</span></code></pre>
<p>For the love of TDD, lets define the unit test.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Test::More</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Exception</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Calculator</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$calc</span> <span class="operator">=</span> <span class="word">Calculator</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /><span class="word">is</span><span class="structure">(</span><span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'add'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">4</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span><span class="operator">,</span> <span class="number">6</span><span class="structure">);</span><br /><span class="word">is</span><span class="structure">(</span><span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'sub'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">4</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span><span class="operator">,</span> <span class="number">2</span><span class="structure">);</span><br /><span class="word">is</span><span class="structure">(</span><span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'mul'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">4</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span><span class="operator">,</span> <span class="number">8</span><span class="structure">);</span><br /><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'add'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Missing required parameter. \(a\)/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'add'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Missing required parameter. \(b\)/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'x'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint. \(op\)/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'add'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="single">'x'</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint. \(a\)/</span><span class="structure">;</span><br /><span class="word">throws_ok</span> <span class="structure">{</span> <span class="symbol">$calc</span><span class="operator">-></span><span class="word">calc</span><span class="structure">({</span> <span class="word">op</span> <span class="operator">=></span> <span class="single">'add'</span><span class="operator">,</span> <span class="word">a</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span> <span class="word">b</span> <span class="operator">=></span> <span class="single">'x'</span> <span class="structure">})</span> <span class="structure">}</span> <span class="regexp">qr/Parameter failed check constraint. \(b\)/</span><span class="structure">;</span><br /><br /><span class="word">done_testing</span><span class="structure">();</span></code></pre>
<h3 id="Conclusion">Conclusion</h3>
<p>I have used <a href="https://metacpan.org/module/Method::ParamValidator">Method::ParamValidator</a> to one of my Google API interface <a href="https://metacpan.org/module/WWW::Google::Places">WWW::Google::Places</a>. Any help to extend the validator would be highly appreciated. Or if you have any suggestions please raise them at <a href="https://github.com/manwar/Method-ParamValidator">GitHub</a>.</p>
</div>2018-12-22T00:00:00ZMohammad S AnwarDebug Hardhttp://perladvent.org/2018/2018-12-21.html<div class='pod'><p>"Maybe I should just give up and go work in the TOY FACTORY!", Snowdrop Cookiefoot shouted as he picked up his laptop. He was just about to throw it through the window when he noticed The Wise Old Elf watching from his office door.</p>
<p>"Problems with your code, Mr Cookiefoot?" he gently enquired.</p>
<p>"Yeah. Sorry Wise Old Elf. It's this code. It's just, you know <i>getting to me</i>. I just can't work out why it's doing what it's doing."</p>
<p>"Ah, you need a good debugger."</p>
<h3 id="Debugging-via-the-Browser">Debugging via the Browser</h3>
<p>There are many different choices for debuggers on Perl, each with their own strengths and weaknesses. The Wise Old Elf had used them all, but given Snowdrop's, um, <i>perilous state of mind</i> he decided he'd better show him one with a super friendly user interface with minimal learning curve.</p>
<p><a href="https://metacpan.org/module/Devel::hdb">Devel::hdb</a> is a Perl debugger which uses a web browser for its front end. When it activates it starts up a web server which you can connect to and immediately see what's going on.</p>
<p>To use it first you need to invoke your program with the <code>-d</code> flag, using the colon syntax to pass <code>hdb</code> to tell it to load <code>Devel::hdb</code>.</p>
<pre><code> perl -d:hdb deliver-presents.pl
Debugger pid 86416 listening on http://127.0.0.1:8080/debugger-gui</code></pre>
<p>You can see that it's printed out a URL for you to visit in the browser. One of the key advantages in Devel::hdb is that since it uses simple HTTP it's really easy to access the web page based debugger UI on a remote machine, and since proxying or tunneling the HTTP protocol is commonplace and widely understood you can even do this behind firewalls.</p>
<p>The interface it loads in the browser is straightforward compared to esoteric command line interfaces offered by the inbuilt debugger and many of the other console debuggers available for Perl.</p>
<p><center><img src="hdb.jpg" width="470" height="450"></center>
</p>
<p><ul> <li>On the left hand side of the screen we have a stack trace. Right now we can see that our main code called run, which called initialize_minicpan, which called the read_config method we're currently displaying. Hovering over these shows us line numbers and clicking the links will show the calling line in the middle panel.</li> <li>In the middle panels is the code we're currently executing (with the blue highlight indicating the current line). We can manually set breakpoints on a per-line basis for any line that has a statement on it by clicking on any uncrossed line number, turning it red like 754 is in the screenshot. We can hover over any variable on screen to see what the current value is. We can use the tab bar to switch between and open new source files to set breakpoints in other files.</li> <li>On the right hand side we can set watch expressions. We can enter the name of any variable that we want to watch and break as soon as the value changes.</li> <li>At the top of the screen are a set of buttons. We can click "Step Over" to move to the next statement on screen, "Step In" to debug further inside a statement by following the subroutines it calls, "Step Out" to run till the current subroutine ends, and "Run" to execute until the next breakpoint or watched expression change occurs.</li> </ul>
</p>
<h3 id="A-Ongoing-Diliemma">A Ongoing Diliemma</h3>
<p>It was a week later when the Wise Old Elf heard the distinctive sound of a laptop soaring through the air and shattering on the ice shelf. Slowly, shaking his head, he walked into Cookiefoot's office.</p>
<p>"I take it the debugger didn't work out?"</p>
<p>"Not really Wise Old Elf", Snowdrop agreed, "At first it was great but soon the whole thing became tedious. I'd need to do the same things over and over again. Or I'd have something really complex and the web page just wasn't up to the job"</p>
<p>"Well, when you get yourself a new laptop, you might want to take a look at the API documentation"</p>
<h3 id="Scripting-Devel::hdb">Scripting Devel::hdb</h3>
<p>The web page interface for hdb is just a JavaScript front end to a bunch of JSON REST endpoints. There's nothing stopping you making those exact same REST calls from a Perl script.</p>
<p>For example, let's print out the current stack information:</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Mojo::UserAgent</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Mojo::Util</span> <span class="words">qw(dumper)</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$ua</span> <span class="operator">=</span> <span class="word">Mojo::UserAgent</span><span class="operator">-></span><span class="word">new</span><span class="structure">();</span><br /><span class="keyword">my</span> <span class="symbol">$response</span> <span class="operator">=</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">get</span><span class="structure">(</span><span class="single">'http://localhost:8080/stack'</span><span class="structure">)</span><span class="operator">-></span><span class="word">res</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$stack</span> <span class="operator">=</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">json</span><span class="structure">;</span><br /><span class="word">print</span> <span class="word">dumper</span> <span class="symbol">$stack</span><span class="structure">;</span></code></pre>
<p>Which generates:</p>
<pre><code class="code-listing"><span class="structure">[</span><br /><span class="structure">{</span><br /> <span class="double">"args"</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="double">"CPAN::Mini::App"</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="double">"autoload"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"bitmask"</span> <span class="operator">=></span> <span class="double">"UUUUUUUUUUUUUUUUUU"</span><span class="operator">,</span><br /> <span class="double">"callsite"</span> <span class="operator">=></span> <span class="single">'140550668291808'</span><span class="operator">,</span><br /> <span class="double">"evalfile"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"evalline"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"evaltext"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"filename"</span> <span class="operator">=></span> <span class="double">"/opt/adventperl/lib/site_perl/5.28.1/CPAN/Mini/App.pm"</span><span class="operator">,</span><br /> <span class="double">"hasargs"</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="double">"hints"</span> <span class="operator">=></span> <span class="number">1762</span><span class="operator">,</span><br /> <span class="double">"href"</span> <span class="operator">=></span> <span class="double">"/stack/0"</span><span class="operator">,</span><br /> <span class="double">"is_require"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"level"</span> <span class="operator">=></span> <span class="number">7</span><span class="operator">,</span><br /> <span class="double">"line"</span> <span class="operator">=></span> <span class="number">59</span><span class="operator">,</span><br /> <span class="double">"package"</span> <span class="operator">=></span> <span class="double">"CPAN::Mini::App"</span><span class="operator">,</span><br /> <span class="double">"serial"</span> <span class="operator">=></span> <span class="number">2089</span><span class="operator">,</span><br /> <span class="double">"subname"</span> <span class="operator">=></span> <span class="double">"initialize_minicpan"</span><span class="operator">,</span><br /> <span class="double">"subroutine"</span> <span class="operator">=></span> <span class="double">"CPAN::Mini::App::initialize_minicpan"</span><span class="operator">,</span><br /> <span class="double">"wantarray"</span> <span class="operator">=></span> <span class="double">""</span><br /><span class="structure">}</span><span class="operator">,</span><br /><span class="structure">{</span><br /> <span class="double">"args"</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="double">"CPAN::Mini::App"</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="double">"autoload"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"bitmask"</span> <span class="operator">=></span> <span class="double">"UUUUUUUUUUUUUUUUUU"</span><span class="operator">,</span><br /> <span class="double">"callsite"</span> <span class="operator">=></span> <span class="single">'140550668347840'</span><span class="operator">,</span><br /> <span class="double">"evalfile"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"evalline"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"evaltext"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"filename"</span> <span class="operator">=></span> <span class="double">"/opt/adventperl/lib/site_perl/5.28.1/CPAN/Mini/App.pm"</span><span class="operator">,</span><br /> <span class="double">"hasargs"</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="double">"hints"</span> <span class="operator">=></span> <span class="number">2018</span><span class="operator">,</span><br /> <span class="double">"href"</span> <span class="operator">=></span> <span class="double">"/stack/1"</span><span class="operator">,</span><br /> <span class="double">"is_require"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"level"</span> <span class="operator">=></span> <span class="number">8</span><span class="operator">,</span><br /> <span class="double">"line"</span> <span class="operator">=></span> <span class="number">47</span><span class="operator">,</span><br /> <span class="double">"package"</span> <span class="operator">=></span> <span class="double">"CPAN::Mini::App"</span><span class="operator">,</span><br /> <span class="double">"serial"</span> <span class="operator">=></span> <span class="number">2088</span><span class="operator">,</span><br /> <span class="double">"subname"</span> <span class="operator">=></span> <span class="double">"run"</span><span class="operator">,</span><br /> <span class="double">"subroutine"</span> <span class="operator">=></span> <span class="double">"CPAN::Mini::App::run"</span><span class="operator">,</span><br /> <span class="double">"wantarray"</span> <span class="operator">=></span> <span class="core">undef</span><br /><span class="structure">}</span><span class="operator">,</span><br /><span class="operator">...</span></code></pre>
<p>Or, we could write the same thing with a one liner with <a href="https://metacpan.org/module/ojo">ojo</a>.</p>
<pre><code> perl -Mojo -E 'print r g("http://localhost:8080/stack")->json'</code></pre>
<p>We're not limited to just reading state, we can make a JSON post request to set a breakpoint:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">ojo</span><span class="structure">;</span><br /><span class="word">p</span><span class="structure">(</span><span class="single">'http://localhost:8080/breakpoints'</span><span class="operator">,</span> <span class="word">json</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="double">"code"</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="double">"filename"</span> <span class="operator">=></span> <span class="double">"/opt/adventperl/lib/site_perl/5.28.1/CPAN/Mini/App.pm"</span><span class="operator">,</span><br /> <span class="double">"inactive"</span> <span class="operator">=></span> <span class="core">undef</span><span class="operator">,</span><br /> <span class="double">"line"</span> <span class="operator">=></span> <span class="number">64</span><br /><span class="structure">});</span></code></pre>
<p>Step in or Step over, Run, etc.</p>
<pre><code> perl -Mojo -E 'p("http://localhost:8080/stepin")'
perl -Mojo -E 'p("http://localhost:8080/stepover")'
perl -Mojo -E 'p("http://localhost:8080/continue")'</code></pre>
<p>Or even evaluate code in the context of the program we're debugging by POSTing JSON and then parsing the response JSON.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">ojo</span><span class="structure">;</span><br /><span class="word">print</span> <span class="word">r</span> <span class="word">p</span><span class="structure">(</span><span class="single">'http://localhost:8080/eval'</span><span class="operator">,</span> <span class="word">json</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="double">"wantarray"</span> <span class="operator">=></span> <span class="number">0</span><span class="operator">,</span><br /> <span class="double">"code"</span> <span class="operator">=></span> <span class="heredoc"><<'PERL'</span><span class="operator">,</span><br /><span class="heredoc_content"> {<br /> random_number => rand(),<br /> date => scalar(gmtime)<br /> }<br /></span><span class="heredoc_terminator">PERL<br /></span><span class="structure">})</span><span class="operator">-></span><span class="word">json</span><span class="structure">(</span><span class="single">'/__value'</span><span class="structure">);</span></code></pre>
<p>Which prints out:</p>
<pre><code class="code-listing"><span class="structure">{</span><br /> <span class="double">"date"</span> <span class="operator">=></span> <span class="double">"Thu Dec 21 02:17:03 2018"</span><span class="operator">,</span><br /> <span class="double">"random_number"</span> <span class="operator">=></span> <span class="double">"0.494246470413"</span><br /><span class="structure">}</span></code></pre>
<p>The API is - obviously - able to do anything the web front end is. It can get the current value of variables, it can set watchpoints or actions to be triggered when a line number and expression match. It can get metainfo about packages and the source code installed on the machine the debugger is running on.</p>
<p>In short, if Snowdrop Cookiefoot hasn't thrown his laptop out of the window he could have easily scripted the debugger to do whatever he wanted.</p>
</div>2018-12-21T00:00:00ZMark FowlerChristmas-iSHhttp://perladvent.org/2018/2018-12-20.html<div class='pod'><p>A lightning strike! A lightning strike on the sleigh!</p>
<p>In all the years that Santa had been delivering presents to Children around the world he'd never been struck by lightning. Until today.</p>
<p>Santa patted out his slightly smoldering beard and looked around to assess the damage. Every single bit of electronics built into the sleigh had been fried. The only bit of technology on the sleigh that still worked was the trusty iPhone that he kept in his pocket.</p>
<p>"Siri! Call The Wise Old Elf"</p>
<p>Santa explained his situation - without the systems built into the sleigh how was he going to run the Perl script the elves had developed to match up presents from the sack with the address he was currently at? With the spotty cell reception throughout the world it's not something that the Wise Old Elf could run in the cloud for him.</p>
<p>"Well your jolliness, there's nothing for it", the Wise Old Elf explained, "We're going to have to run Perl on your iPhone."</p>
<h3 id="iSH">iSH</h3>
<p>The iPhone isn't exactly well known for being an open platform. Unlike a Linux based Android phone you're not able to ssh into it and just install perl unless you jailbreak the device. A security conscious organization like the North Pole just wasn't willing to go off-book with their devices like that.</p>
<p>What the Wise Old Elf was going to instead do was have Santa install an app called <a href="https://ish.app/">iSH</a>, the Linux shell on iOS.</p>
<p>iSH is an app that uses a usermode x86 emulator to run a Linux environment just as any other executable might run on the phone. This crazy app is still very much in beta - it's not available on the App Store like a normal app, and if you want to install it on your iPhone you're going to have to take some unusual steps.</p>
<p>As you might expect for such a project, the source code is <a href="https://github.com/tbodt/ish/">available on Github</a>, and you can compile it yourself, use a free developer account to sign it, and upload it to your own devices.</p>
<p>Another simpler approach is to join the TestFlight beta program for iSH directly from the link on the iSH homepage. This will allow you to download a beta releases of the application with Apple's TestFlight application "over the air" - no computer required. Which is pretty handy if you're currently in a slightly charred sleigh on Christmas Eve.</p>
<p>Once you launch the iSH app you get terminal access to a bare-bones <a href="https://alpinelinux.org/">Alpine Linux</a> environment. From here you can use the package manager to install perl, and then you can run Perl code on your phone from the command line!</p>
<p><center><img src="iPhone.jpg" width="460" height="900" alt="An iPhone running iSH"></center>
</p>
<p>You can also install all that you need to build Perl modules (make, gcc, wget, etc) and tools to access files over the network (Santa can install scp and download the scripts from the North Pole server with it.)</p>
<p>It might be a little rough around the edges currently, but iSH should allow Santa to complete his deliveries this year - and who knows, with the rapid pace of development it might be on the App Store by next year, just in case lightning does strike twice in the same place!</p>
</div>2018-12-20T00:00:00ZMark FowlerCute Christmas Animalshttp://perladvent.org/2018/2018-12-19.html<div class='pod'><p>State sponsored hacking. Influencing foreign elections. Crypto currency ponzi schemes. Have we forgotten what the Internet is really for? <i>Looking at cute pictures of cats</i>.</p>
<p><center><img src="https://i.imgur.com/XgJetkM.jpg" width="500"></center>
</p>
<p>With all the craziness that's going on online, it feels good to take a break from it all and just look at pictures of cats for a bit. And you know what can help with that? Yep...Perl!</p>
<p>Let's write a quick Perl script to get us some <i>Christmas</i> cat pictures to look at by accessing Imgur's REST API.</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Mojo::UserAgent</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Mojo::Template</span><span class="structure">;</span><br /><br /><span class="comment"># get your own client id at https://api.imgur.com/oauth2/addclient<br /></span><span class="keyword">my</span> <span class="symbol">$CLIENT_ID</span> <span class="operator">=</span> <span class="single">'<<REDACTED>>'</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$results</span> <span class="operator">=</span> <span class="word">Mojo::UserAgent</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">get</span><span class="structure">(</span><br /> <span class="word">Mojo::URL</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="single">'https://api.imgur.com'</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">path</span><span class="structure">(</span><span class="single">'/3/gallery/search'</span><span class="structure">)</span><br /> <span class="operator">-></span><span class="word">query</span><span class="structure">(</span><br /> <span class="word">q_all</span> <span class="operator">=></span> <span class="single">'christmas cat'</span><span class="operator">,</span><br /> <span class="word">q_not</span> <span class="operator">=></span> <span class="single">'tag:"secret santa"'</span><span class="operator">,</span><br /> <span class="structure">)</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">Authorization</span> <span class="operator">=></span> <span class="double">"Client-ID $CLIENT_ID"</span><span class="structure">}</span><span class="operator">,</span><br /><span class="structure">)</span><span class="operator">-></span><span class="word">result</span><span class="operator">-></span><span class="word">json</span><span class="structure">(</span><span class="single">'/data'</span><span class="structure">);</span><br /><br /><span class="keyword">my</span> <span class="symbol">$mt</span> <span class="operator">=</span> <span class="word">Mojo::Template</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><span class="word">print</span> <span class="symbol">$mt</span><span class="operator">-></span><span class="word">vars</span><span class="structure">(</span><span class="number">1</span><span class="structure">)</span><span class="operator">-></span><span class="word">render</span><span class="structure">(</span><span class="heredoc"><<'HTML'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">data</span> <span class="operator">=></span> <span class="symbol">$results</span> <span class="structure">});</span><br /><span class="heredoc_content"><html><br /><body><br />% for my $post (@{ $data }) {<br /><h2><%= $post->{title} %></h2><br />% for my $image (@{ $post->{images} }) {<br /><img src="<%= $image->{link} %>" width="400"><br /><p><%= $image->{description} || ""%></p><br />% }<br />% }<br /></body><br /></html><br /></span><span class="heredoc_terminator">HTML<br /></span></code></pre>
<p>Rather than using LWP::UserAgent and URI (like in <a href="http://perladvent.org/2018/2018-12-17.html">the Spotify REST API example</a>) we're using Mojolicious's <a href="https://metacpan.org/module/Mojo::UserAgent">Mojo::UserAgent</a> and <a href="https://metacpan.org/module/Mojo::URL">Mojo::URL</a> to make the request and build the URL. Mojolicious's interface is a little more compact, allowing us to easily do things like chain method calls as we're building the URL object (with each mutating call returning the object itself again) so we can construct it directly inside our user agent method call. With Mojolicious there's no need to explicitly create a request object in order to simply set a header. The JSON parsing is also handled as part of the Mojolicious framework with a call to the <code>json</code> method rather than needing to make use of a seperate <code>decode_json</code> function call - in addition, we're passing it a <a href="https://tools.ietf.org/html/rfc6901">JSON Pointer</a> to indicate which <i>bit</i> of the JSON data structure it should return.</p>
<p>All in all, it's pretty neat having to write less code. But the real advantages in using Mojo to do this comes when you want to start doing concurrency...</p>
<h3 id="What-about-the-Dogs">What about the Dogs?</h3>
<p>What you say? You're a <i>Dog</i> person? Well, me too. Our house is blessed by not only two cute cats but a wonderful basset hound (who fight each other like cats and dogs...but I digress.) So, yep, I like looking at pictures of dogs too. Maybe we can adjust our script to show pictures of Christmas dogs as well?</p>
<p><center><img src="https://i.imgur.com/gGiXuyv.jpg" width="500"></center>
</p>
<p>What I really want to do is alternate posts from the cat search with posts from the dog search. But to do that I need to make two API calls, and I'm an <i>incredibly</i> impatient person when it comes to getting my cute animal fix. I don't want to wait for my cute Christmas cats API call to return before making my cute Christmas dogs API call. What I want to do is make them at <i>the same time</i>.</p>
<p>Instead of returning a result we could have Mojo::UserAgent use the Mojo IO loop to fire a callback when it's complete. This is achieved by passing an anonymous subroutine as an argument:</p>
<pre><code class="code-listing"><span class="symbol">$ua</span><span class="operator">-></span><span class="word">get</span><span class="structure">(</span><br /> <span class="word">Mojo::URL</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="single">'https://api.imgur.com'</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">path</span><span class="structure">(</span><span class="single">'/3/gallery/search'</span><span class="structure">)</span><br /> <span class="operator">-></span><span class="word">query</span><span class="structure">(</span><br /> <span class="word">q_all</span> <span class="operator">=></span> <span class="single">'christmas $animal'</span><span class="operator">,</span><br /> <span class="word">q_not</span> <span class="operator">=></span> <span class="single">'tag:"secret santa"'</span><span class="operator">,</span><br /> <span class="structure">)</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">Authorization</span> <span class="operator">=></span> <span class="double">"Client-ID $CLIENT_ID"</span><span class="structure">}</span><span class="operator">,</span><br /> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$ua</span><span class="operator">,</span> <span class="symbol">$tx</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"Got data back for $animal...\n"</span><span class="structure">;</span><br /><br /> <span class="operator">...</span><br /> <span class="structure">}</span><br /><span class="structure">);</span></code></pre>
<p>But what are we going to replace the <code>...</code> with? What we need is some way to await the content of both of the REST API calls we're fetching and <i>then</i> do something. The easiest way to do this is to use Mojolicious' implementation of promises: <a href="https://metacpan.org/module/Mojo::Promise">Mojo::Promise</a>.</p>
<p>When we write our <code>perform_search</code> function rather than waiting on the API call and only then returning from the function we instead immediately return a new <i>promise</i> object, a promise to later return the API data.</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">perform_search</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$animal</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /><span class="comment"> # create a new promise to return<br /></span> <span class="keyword">my</span> <span class="symbol">$promise</span> <span class="operator">=</span> <span class="word">Mojo::Promise</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">get</span><span class="structure">(</span><br /> <span class="word">Mojo::URL</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="single">'https://api.imgur.com'</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">path</span><span class="structure">(</span><span class="single">'/3/gallery/search'</span><span class="structure">)</span><br /> <span class="operator">-></span><span class="word">query</span><span class="structure">(</span><br /> <span class="word">q_all</span> <span class="operator">=></span> <span class="double">"christmas $animal"</span><span class="operator">,</span><br /> <span class="word">q_not</span> <span class="operator">=></span> <span class="single">'tag:"secret santa"'</span><span class="operator">,</span><br /> <span class="structure">)</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">Authorization</span> <span class="operator">=></span> <span class="double">"Client-ID $CLIENT_ID"</span><span class="structure">}</span><span class="operator">,</span><br /> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$ua</span><span class="operator">,</span> <span class="symbol">$tx</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"Got data back for $animal...\n"</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$data</span> <span class="operator">=</span> <span class="symbol">$tx</span><span class="operator">-></span><span class="word">result</span><span class="operator">-></span><span class="word">json</span><span class="structure">(</span><span class="single">'/data'</span><span class="structure">);</span><br /><br /><span class="comment"> # resolve the promise with the data now we have it<br /></span> <span class="symbol">$promise</span><span class="operator">-></span><span class="word">resolve</span><span class="structure">(</span> <span class="symbol">$data</span> <span class="structure">);</span><br /> <span class="structure">}</span><br /> <span class="structure">);</span><br /><br /><span class="comment"> # immediately return the promise object<br /></span> <span class="keyword">return</span> <span class="symbol">$promise</span><span class="structure">;</span><br /><span class="structure">}</span></code></pre>
<p>When the callback is executed it calls the <code>resolve</code> method on the promise meaning anything waiting on that promise will be allowed to continue. For example:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$promise</span> <span class="operator">=</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">);</span><br /><span class="symbol">$promise</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span></code></pre>
<p><code>wait</code> runs the Mojolicious IO loop and halts the current execution flow until the promise is fulfilled or fails. In our example this happens when <code>resolve</code> is called by the <code>get</code> method's callback when the REST API response arrives - meaning <code>wait</code> in this example effectively stops the program until the API result has been downloaded.</p>
<p>Great! So how do we collect the promised result? The easiest way to do this is to use the <code>then</code> method on the promise. You can pass an anonymous subroutine to <code>then</code> which will be executed when that promise is resolved with whatever value was used to resolve it. Here's the really clever bit though: when called <code>then</code> itself immediately returns a new promise. This new promise will only get itself resolved when whatever happens in the anonymous subroutine is done.</p>
<p>If we want to capture our cat data structure we could use a <code>then</code>/<code>wait</code> pair like so:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$cat_results</span><span class="structure">;</span><br /><span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="structure">(</span><span class="symbol">$cat_results</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="keyword">return</span><span class="structure">;</span><br /><span class="structure">})</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span></code></pre>
<p>The anonymous subroutine is called when the promise that <code>perform_search</code> created is resolved (when the API request has returned). When that anonymous subroutine returns the promise that <code>then</code> created which we're waiting on is resolved. By creating this chain we're able to control the program flow.</p>
<p>Above I mentioned that the promise that <code>then</code> returns is resolved when the subroutine is done...but what did I mean by that? It's not <i>just</i> whenever the subroutine returns a value. The really really clever bit is that this occurs <i>either</i> when the anonymous subroutine returns a non promise value <i>or</i> it returns a promise that itself resolves! For example, we could return a promise for the dog API call in the first <code>then</code> subroutine and then the promise created will only allow us to move onto the next <code>then</code> when the dog search API promise is resolved. This sounds complicated, but actually is very readable in code:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$cat_results</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$dog_results</span><span class="structure">;</span><br /><br /><span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="structure">(</span><span class="symbol">$cat_results</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'dog'</span><span class="structure">);</span><br /><span class="structure">})</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="structure">(</span><span class="symbol">$dog_results</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><br /><span class="structure">})</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span></code></pre>
<p>Before we continue onto running these requests in parallel we should do a little housekeeping. Mojo::UserAgent is actually able to create promises directly with the <code>get_p</code> convenience method. Now we know how to use <code>then</code> we can simplify our <code>perform_search</code> function greatly by chaining our promises within <code>perform_search</code>:</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">perform_search</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$animal</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">return</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">get_p</span><span class="structure">(</span><br /> <span class="word">Mojo::URL</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="single">'https://api.imgur.com'</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">path</span><span class="structure">(</span><span class="single">'/3/gallery/search'</span><span class="structure">)</span><br /> <span class="operator">-></span><span class="word">query</span><span class="structure">(</span><br /> <span class="word">q_all</span> <span class="operator">=></span> <span class="double">"christmas $animal"</span><span class="operator">,</span><br /> <span class="word">q_not</span> <span class="operator">=></span> <span class="single">'tag:"secret santa"'</span><span class="operator">,</span><br /> <span class="structure">)</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">Authorization</span> <span class="operator">=></span> <span class="double">"Client-ID $CLIENT_ID"</span><span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$tx</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="symbol">$tx</span><span class="operator">-></span><span class="word">result</span><span class="operator">-></span><span class="word">json</span><span class="structure">(</span><span class="single">'/data'</span><span class="structure">);</span><br /> <span class="structure">});</span><br /><span class="structure">}</span></code></pre>
<p>We're almost there! Now can we finally run these in parallel? You bet! There's no reason we have to pause for the cat promise to complete before retrieving the dog promise:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$cat_results</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$dog_results</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$promise1</span> <span class="operator">=</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span> <span class="symbol">$cat_results</span> <span class="operator">=</span> <span class="core">shift</span> <span class="structure">});</span><br /><span class="keyword">my</span> <span class="symbol">$promise2</span> <span class="operator">=</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'dog'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span> <span class="symbol">$dog_results</span> <span class="operator">=</span> <span class="core">shift</span> <span class="structure">});</span><br /><span class="symbol">$promise1</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span><br /><span class="symbol">$promise2</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Data::Dumper</span><span class="structure">;</span><br /><span class="word">print</span> <span class="word">Dumper</span> <span class="structure">[</span><span class="symbol">$cat_results</span><span class="operator">,</span> <span class="symbol">$dog_results</span><span class="structure">];</span></code></pre>
<p>Looking good! Mojo::Promise's <code>all</code> method makes this even easier, allowing us to quickly create a single promise that will resolve when all the promises we pass to it themselves resolve:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$cat_results</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$dog_results</span><span class="structure">;</span><br /><span class="word">Mojo::Promise</span><span class="operator">-></span><span class="word">all</span><span class="structure">(</span><br /> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span> <span class="symbol">$cat_results</span> <span class="operator">=</span> <span class="core">shift</span> <span class="structure">})</span><span class="operator">,</span><br /> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'dog'</span><span class="structure">)</span><span class="operator">-></span><span class="word">then</span><span class="structure">(</span><span class="keyword">sub</span> <span class="structure">{</span> <span class="symbol">$dog_results</span> <span class="operator">=</span> <span class="core">shift</span> <span class="structure">})</span><span class="operator">,</span><br /><span class="structure">)</span><span class="operator">-></span><span class="word">wait</span><span class="structure">;</span></code></pre>
<p>Or, even more simply necessitating only writing one <code>then</code> call:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$cat_results</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$dog_results</span><span class="structure">;</span><br /><span class="word">Mojo::Promise</span><br /> <span class="operator">-></span><span class="word">all</span><span class="structure">(</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'cat'</span><span class="structure">)</span><span class="operator">,</span> <span class="word">perform_search</span><span class="structure">(</span><span class="single">'dog'</span><span class="structure">)</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">then</span><span class="structure">(</span> <span class="keyword">sub</span> <span class="structure">{</span> <span class="structure">(</span><span class="symbol">$cat_results</span><span class="operator">,</span> <span class="symbol">$dog_results</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span> <span class="structure">}</span> <span class="structure">)</span><br /> <span class="operator">-></span><span class="word">wait</span><span class="structure">;</span></code></pre>
<p>Now all that's left to do is zip the results together so we get alternating cat and dog posts (we can even use the same template code we had before):</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">List::MoreUtils</span> <span class="words">qw(zip)</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$results</span> <span class="operator">=</span> <span class="structure">[</span> <span class="word">zip</span> <span class="cast">@</span><span class="structure">{</span> <span class="symbol">$cat_results</span> <span class="structure">}</span><span class="operator">,</span> <span class="cast">@</span><span class="structure">{</span> <span class="symbol">$dog_results</span> <span class="structure">}</span> <span class="structure">];</span><br /><br /><span class="keyword">my</span> <span class="symbol">$mt</span> <span class="operator">=</span> <span class="word">Mojo::Template</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><span class="word">print</span> <span class="symbol">$mt</span><span class="operator">-></span><span class="word">vars</span><span class="structure">(</span><span class="number">1</span><span class="structure">)</span><span class="operator">-></span><span class="word">render</span><span class="structure">(</span><span class="heredoc"><<'HTML'</span><span class="operator">,</span> <span class="structure">{</span> <span class="word">data</span> <span class="operator">=></span> <span class="symbol">$results</span> <span class="structure">});</span><br /><span class="heredoc_content"><html><br /><body><br />% for my $post (@{ $data }) {<br /><h2><%= $post->{title} %></h2><br />% for my $image (@{ $post->{images} }) {<br /><img src="<%= $image->{link} %>" width="400"><br /><p><%= $image->{description} || ""%></p><br />% }<br />% }<br /></body><br /></html><br /></span><span class="heredoc_terminator">HTML<br /></span></code></pre>
<p>And there, as promised (groan) is our Christmas Cat and Dog Extravaganza.</p>
</div>2018-12-19T00:00:00ZMark FowlerTest/Compare Your Excelhttp://perladvent.org/2018/2018-12-18.html<div class='pod'><p>Santa's elves have to deal with a wide range of suppliers throughout the year in order to get all the parts for the toys they assemble at the North Pole. Each one of these seems to have their own particular workflow.</p>
<p>Spruce Glowchaser was dealing with a supplier that needed an Excel spreadsheet filled out and emailed to them. Over the last few weeks he'd painstakingly written the automation code to generate the spreadsheet using <a href="https://metacpan.org/module/Spreadsheet::WriteExcel">Spreadsheet::WriteExcel</a>. He <i>thought</i> he'd managed to faithfully reproduce the example sheet from the supplier - that was until the QA department had kicked back his output with numerous bugs.</p>
<p>"What you need to do", the Wise Old Elf had chagrined him, "is write some tests"</p>
<h2 id="Introducing-Test::Excel">Introducing Test::Excel</h2>
<p><a href="https://metacpan.org/module/Test::Excel">Test::Excel</a> was first published to CPAN on 13th Aug 2010, a module for visual comparison of Excel files. Test::Excel was my first contribution ever and is very close to my heart.</p>
<p>When understanding how Test::Excel works it is important to understand the term "visually". The comparisons it offers compare the data structure in each of the Excel files and ignore components such as embedded fonts, images, forms and annotations.</p>
<p>The package Test::Excel exports the following functions:</p>
<dl>
<dt>cmp_excel_ok($got, $exp, \%$rule, $message)</dt>
<dd>
<p>The params <code>$got</code> and <code>$exp</code> can either be a path to the two Excel files being compared or they can each be an object of type <a href="https://metacpan.org/module/Spreadsheet::Read">Spreadsheet::Read</a>. The param <code>\%rule</code> is optional. The final param <code>$message</code> is the test message that will be associated with the test when its run (which is optional as well). The function is ideal for use in unit test as below:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span> <span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::More</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Excel</span><span class="structure">;</span><br /><br /><span class="comment"># do the two files foo.xls and bar.xls contain the same data?<br /></span><span class="word">cmp_excel_ok</span><span class="structure">(</span><span class="single">'foo.xls'</span><span class="operator">,</span> <span class="single">'bar.xls'</span><span class="structure">);</span><br /><br /><span class="word">done_testing</span><span class="structure">();</span></code></pre>
<p>Test::Excel also exports a <code>cmp_excel()</code> function that acts identially to <code>cmp_excel_ok()</code>.</p>
</dd>
<dt>cmp_execel_not_ok($got, $exp, \%$rule, $message)</dt>
<dd>
<p>This function is just the reverse of <code>cmp_excel_ok()</code> - it passes if the data structures in the two excel files don't match each other. It is ideal for use in unit test as below:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span> <span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::More</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Excel</span><span class="structure">;</span><br /><br /><span class="comment"># do the two files foo.xls and bar.xls contain different data?<br /></span><span class="word">cmp_excel_not_ok</span><span class="structure">(</span><span class="single">'foo.xls'</span><span class="operator">,</span> <span class="single">'bar.xls'</span><span class="structure">);</span><br /><br /><span class="word">done_testing</span><span class="structure">();</span></code></pre>
</dd>
<dt>compare_excel($got, $exp, \%$rule)</dt>
<dd>
<p>This is the core function that does all the underlying comparisons that <code>cmp_excel_ok</code> and <code>cmp_excel_not_ok</code> rely upon. It can be used as standalone function outside of test environment and doesn't create the <code>ok 1</code> type test output that the previous functions do.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span> <span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Test::Excel</span><span class="structure">;</span><br /><br /><span class="word">print</span> <span class="double">"Excel files are identical.\n"</span> <span class="word">if</span> <span class="word">compare_excel</span><span class="structure">(</span><span class="double">"foo.xls"</span><span class="operator">,</span> <span class="double">"bar.xls"</span><span class="structure">);</span></code></pre>
</dd>
</dl>
<h3 id="Spruces-Dilemma">Spruce's Dilemma</h3>
<p>"I did try to use that module", Spruce explained to the Wise Old Elf, "but it didn't work for me."</p>
<p>"Oh?"</p>
<p>"The spreadsheet I'm producing isn't exactly the same as the test example. It only has to be identical in these particular cells on this particular page - everything else can change."</p>
<p>"Did you look at the optional <code>%rule</code> argument?", the Wise Old Elf questioned.</p>
<h3 id="Rules-Specification">Rules Specification</h3>
<p>If you noticed above, all the methods accept an optional parameter <code>\%rule</code>. So what is this all about? You can finetune the comparison rule used by the comparison functions to control exactly what must match, and how it must match.</p>
<dl>
<dt>spec</dt>
<dd>
<p>Path to specification file that can control which parts of the sheet match. With this you can define which ranges will match.</p>
</dd>
<dt>sheet</dt>
<dd>
<p>The name of the sheet that the module should compare. Multiple sheets can be specified in a single string by using a <code>|</code> delimiter.</p>
</dd>
<dt>tolerance</dt>
<dd>
<p>Tolerance amount acceptable. Applicable to all except on 'sheet' or 'spec'</p>
</dd>
<dt>sheet_tolerance</dt>
<dd>
<p>Tolerance amount acceptable. Applicable only to 'sheet' or 'range' in the spec file.</p>
</dd>
<dt>swap_check</dt>
<dd>
<p>Boolean 0 or 1, whether row swapping check is on or off.</p>
</dd>
<dt>error_limit</dt>
<dd>
<p>Limit the number of errors acceptable. Default is 0.</p>
</dd>
</dl>
<h2 id="The-Specification-File">The Specification File</h2>
<p>The spcification file contains space separated key value pair per line. It can have the following keys:</p>
<dl>
<dt>sheet</dt>
<dd>
<p>Sheet name in the Excel file.</p>
</dd>
<dt>range</dt>
<dd>
<p>Cell range.</p>
</dd>
<dt>ignorerange</dt>
<dd>
<p>Cell range to ignore when comparing.</p>
</dd>
</dl>
<p>For example:</p>
<pre><code> Sheet Parts
Range B3:B5
Sheet Spares
Range B5:B6</code></pre>
<h3 id="It-all-went-wrong">It all went wrong?</h3>
<p>"This test is just failing", Spruce complained, "and I just can't work out why!"</p>
<p>"Did you read the part of the manual about debugging?" the Wise Old Elf asked. "You can set the <code>DEBUG</code> environment variable and you'll get copious output out about what's actually going on under the hood."</p>
<h3 id="DEBUG-output">DEBUG output</h3>
<p>Indeed, setting the DEBUG flag from the command line can be <i>very</i> useful in figuring out what's going on.</p>
<pre><code> bash$ DEBUG=1 perl t/01xls.t
1..6
INFO: Excel comparison [t/got-4.xls] [t/exp-4.xls]
INFO: [MySheet1]:[0][0]:[6][2]
INFO: [MySheet1]:[0][0]:[6][2]
INFO: [STRING]:[MySheet1]:[STD][1][1]:[Name][Name] ... [PASS]
INFO: [STRING]:[MySheet1]:[STD][1][2]:[Value][Value] ... [PASS]
INFO: [STRING]:[MySheet1]:[STD][3][1]:[GBP][GBP] ... [PASS]
INFO: [NUMBER]:[MySheet1]:[SPC][3][2]:[1.2345][1.2345] ... [PASS]
INFO: [STRING]:[MySheet1]:[STD][4][1]:[USD][USD] ... [PASS]
INFO: [NUMBER]:[MySheet1]:[SPC][4][2]:[0.8922][0.9122] ... [PASS]
INFO: [STRING]:[MySheet1]:[STD][5][1]:[INR][INR] ... [PASS]
INFO: [NUMBER]:[MySheet1]:[SPC][5][2]:[1.45][1.567] ... [PASS]
INFO: [STRING]:[MySheet1]:[STD][6][1]:[EUR][EUR] ... [PASS]
INFO: [NUMBER]:[MySheet1]:[STD][6][2]:[2.5][2.5] ... [PASS]
INFO: [MySheet1]: ..... [OK].
INFO: [MySheet2]:[0][0]:[6][2]
INFO: [MySheet2]:[0][0]:[6][2]
INFO: [STRING]:[MySheet2]:[STD][1][1]:[Name][Name] ... [PASS]
INFO: [STRING]:[MySheet2]:[STD][1][2]:[Value][Value] ... [PASS]
INFO: [STRING]:[MySheet2]:[STD][3][1]:[XYZ][XYZ] ... [PASS]
INFO: [NUMBER]:[MySheet2]:[STD][3][2]:[0.12334][0.12334] ... [PASS]
INFO: [STRING]:[MySheet2]:[STD][4][1]:[ABC][ABC] ... [PASS]
INFO: [NUMBER]:[MySheet2]:[STD][4][2]:[1.864][1.864] ... [PASS]
INFO: [STRING]:[MySheet2]:[STD][5][1]:[PQR][PQR] ... [PASS]
INFO: [NUMBER]:[MySheet2]:[SPC][5][2]:[1.99][2.014] ... [PASS]
...</code></pre>
<h2 id="Futher-Examples-and-Contributing">Futher Examples and Contributing</h2>
<p>You can find loads of example in the <a href="https://metacpan.org/pod/Test::Excel">distribution</a>.</p>
<p>If you any suggestions for improvement or if you fing bugs in the code then please raise them at <a href="https://github.com/manwar/Test-Excel">GitHub</a>.</p>
</div>2018-12-18T00:00:00ZMohammad S AnwarChristmas Tunes!http://perladvent.org/2018/2018-12-17.html<div class='pod'><p><center><img src="noddy.jpg" width="315" height="315" alt="Noddy Meme"></center>
</p>
<p>For some people it's just not Christmas until we hear Noddy Holder shouting "It's Christmas!" at the end of Slade's Merry Xmas Everybody. Heck, it's just not Christmas without our Christmas Music. Be it daft old Christmas number ones, covers of carols by warbling modern day divas, or choir recordings, we've all got our own little playlists that put us in the Christmas mood. Honestly, I've listened to the same playlist as the kids and I decorated the tree for the last three years.</p>
<p>Being an Apple geek, I subscribe to Apple Music and that's where <a href="https://itunes.apple.com/us/playlist/marks-christmas-playlist/pl.bdbdd828fdba47778211ce4d248f8ee9">that playlist lives</a> for any other Apple Music subscribers that have my same eccentric taste to listen to.</p>
<p>The trouble with Apple Music is that, unlike some of its competitors, it doesn't have a free ad-supported web player that anyone can click on and immediately listen to the playlist. If I really want to spread Christmas cheer wide I should port that playlist over to Spotify for sharing purposes. Of course, that sounds like a lot of work...unless I get Perl to do it for me!</p>
<h3 id="Interrogating-iTunes">Interrogating iTunes</h3>
<p>The easiest way to extract the details of my playlist from Apple Music is to use the Mac's <i>Open Scripting Architecture</i> to talk directly to iTunes and ask it for the details we want. Far by the most common way to do this on a Mac is to write some AppleScript which sends the OSA events for you automatically. AppleScript is a very odd bespoke programming langauge designed to have a shallow learning curve for non-programmers to do simple things. Unfortunately however, this means it has a very steep learning curve for programmers who are used to totally different syntax when they want to do anything moderately complex (like, say, writing a loop or subroutine.) Luckily being an <i>Open</i> Scripting Architecture means that other languages can send these events just as well - you can even do this with Perl using <a href="https://metacpan.org/module/Mac::Glue">Mac::Glue</a>.</p>
<p>Sometimes however - shock, horror - the best programming language for a task always <i>isn't Perl</i>. Apple produce <a href="https://www.macstories.net/tutorials/getting-started-with-javascript-for-automation-on-yosemite/">JavaScript For Automation</a> which is a simple interface to the OSA from JavaScript. And Perl being Perl it's <i>really</i> easy to glue this in the middle of a Perl script:</p>
<pre><code class="code-listing"><span class="comment">#!perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">JSON::PP</span> <span class="words">qw( encode_json decode_json )</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">IPC::Run3</span> <span class="words">qw( run3 )</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$js</span> <span class="operator">=</span> <span class="heredoc"><<'JAVASCRIPT'</span><span class="structure">;</span><br /><span class="heredoc_content">console.log(<br /> JSON.stringify(<br /> Application('iTunes').currentPlaylist.tracks().map( track => {<br /> return {<br /> artist : track.artist(),<br /> title : track.name()<br /> };<br /> })<br /> )<br />)<br /></span><span class="heredoc_terminator">JAVASCRIPT<br /></span><br /><span class="keyword">my</span> <span class="symbol">$output</span><span class="structure">;</span><br /><span class="word">run3</span><span class="structure">([</span><span class="words">qw( osascript -l JavaScript )</span><span class="structure">]</span><span class="operator">,</span> <span class="cast">\</span><span class="symbol">$js</span><span class="operator">,</span> <span class="core">undef</span><span class="operator">,</span> <span class="cast">\</span><span class="symbol">$output</span> <span class="structure">);</span><br /><span class="keyword">my</span> <span class="symbol">$itunes_tracks</span> <span class="operator">=</span> <span class="word">decode_json</span><span class="structure">(</span><span class="symbol">$output</span><span class="structure">);</span></code></pre>
<p>The technique is simple: Write some JavaScript, have it output JSON. Send that JavaScript on STDIN to <code>osascript</code>, capture the JSON it outputs and decode it in Perl. To give us a data structure that looks something like this:</p>
<pre><code class="code-listing"><span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="single">'artist'</span> <span class="operator">=></span> <span class="single">'Slade'</span><span class="operator">,</span><br /> <span class="single">'title'</span> <span class="operator">=></span> <span class="single">'Merry Xmas Everybody'</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="single">'artist'</span> <span class="operator">=></span> <span class="single">'Vile Richard'</span><span class="operator">,</span><br /> <span class="single">'title'</span> <span class="operator">=></span> <span class="single">'We Wish You a Merry Christmas'</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="single">'artist'</span> <span class="operator">=></span> <span class="single">'The Darkness'</span><span class="operator">,</span><br /> <span class="single">'title'</span> <span class="operator">=></span> <span class="single">'Christmas Time (Don\'t Let the Bells End)'</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="single">'title'</span> <span class="operator">=></span> <span class="single">'Last Christmas (Single Version)'</span><span class="operator">,</span><br /> <span class="single">'artist'</span> <span class="operator">=></span> <span class="single">'Wham!'</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="operator">...</span></code></pre>
<h3 id="Signing-Up-For-a-Spotify-Developer-Account">Signing Up For a Spotify Developer Account</h3>
<p>I'm going to need to use the Spotify Web API to build a playlist full of these tracks. Before I can do that, I need to sign up with their developer platform and create a new test application.</p>
<p>After logging into Spotify and navigating to the <a href="https://developer.spotify.com/dashboard/applications">Spotify Developer Dashboard</a> I can click on "Create a Client ID"</p>
<p><center><img src="spotify1.jpg" width="514" height="368" alt="Spotify Client Registration Picture"></center>
</p>
<p>And then fill in all the details for my account. Since this is just a bit of fun, I just accepted the non-commercial agreements.</p>
<p><center><img src="spotify2.jpg" width="514" height="368" alt="Spotify Client Registration Picture"></center>
</p>
<p>Once I've clicked through I end up on a screen that contains my client id and my client secret. I'll need these later:</p>
<p><center><img src="spotify3.jpg" width="514" height="368" alt="Spotify Client Registration Picture"></center>
</p>
<p>There's one more thing to do before we're done with the web interface. I need to register a callback URL that my Perl code will use in the OAuth process that I'll be describing later. Clicking on the "Edit Settings" page takes me to a page where I can enter the http://localhost:8082/callback address that the module I'm going to introduce in a minute uses:</p>
<p><center><img src="spotify4.jpg" width="514" height="368" alt="Spotify Client Registration Picture"></center>
</p>
<h3 id="OAuth">OAuth</h3>
<p>So far I've only registered a developer application. Now I need to go through the OAuth process that grants my developer application the contexts (the privilidges) it needs to access my account (even though it's only me using this application with my own account, Spotify needs us to go through the same process that we'd use if we were letting many different people use this application with their own accounts.)</p>
<p>A typical OAuth enabled web application bounces you from the application's website to Spotify's website, has you log in, then show you a permissions dialog saying what contexts (privileges) you're prepared to have that application use and then finally bounce you back to the original application's website's callback URL with secrets in the URL parameters. But we're developing a command line application. How is that going to work?</p>
<p>Well, as usual, there's a module for that on the CPAN: <a href="https://metacpan.org/module/OAuth::Cmdline::Spotify">OAuth::Cmdline::Spotify</a>. We're going to use it first in a simple standalone script:</p>
<pre><code class="code-listing"><span class="comment">#!perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">OAuth::Cmdline::Spotify</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">OAuth::Cmdline::Mojo</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$oauth</span> <span class="operator">=</span> <span class="word">OAuth::Cmdline::Spotify</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><br /> <span class="word">client_id</span> <span class="operator">=></span> <span class="single">'**REDACTED**'</span><span class="operator">,</span><br /> <span class="word">client_secret</span> <span class="operator">=></span> <span class="single">'**REDACTED**'</span><span class="operator">,</span><br /> <span class="word">login_uri</span> <span class="operator">=></span> <span class="single">'https://accounts.spotify.com/authorize'</span><span class="operator">,</span><br /> <span class="word">token_uri</span> <span class="operator">=></span> <span class="single">'https://accounts.spotify.com/api/token'</span><span class="operator">,</span><br /> <span class="word">scope</span> <span class="operator">=></span> <span class="word">join</span> <span class="single">','</span><span class="operator">,</span> <span class="words">qw(<br /> playlist-read-private<br /> playlist-modify-private<br /> playlist-modify-public<br /> )</span><br /><span class="structure">);</span><br /><br /><span class="keyword">my</span> <span class="symbol">$app</span> <span class="operator">=</span> <span class="word">OAuth::Cmdline::Mojo</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><br /> <span class="word">oauth</span> <span class="operator">=></span> <span class="symbol">$oauth</span><span class="operator">,</span><br /><span class="structure">);</span><br /><span class="symbol">$app</span><span class="operator">-></span><span class="word">start</span><span class="structure">(</span> <span class="single">'daemon'</span><span class="operator">,</span> <span class="single">'-l'</span><span class="operator">,</span> <span class="symbol">$oauth</span><span class="operator">-></span><span class="word">local_uri</span> <span class="structure">);</span></code></pre>
<p>When this script is executed it starts up a webserver running on localhost:</p>
<pre><code> shell$ perl ~/tmp/oath.pl
[2018-12-09 08:58:07.68615] [32423] [info] Listening at "http://localhost:8082"
Server available at http://localhost:8082</code></pre>
<p>Visiting that URL gives us a simple link to Spotify, allowing me to log in if needed, and then display the permissions dialog:</p>
<p><center><img src="spotify5.jpg" width="700" height="636" alt="Spotify OAuth"></center>
</p>
<p>When I click "Okay" I'll be redirected to the callback URL on localhost that'll capture the token and store it in <code>~/.spotify.yml</code> for later use.</p>
<p>Whenever another script now needs that OAuth token we can use OAuth::Cmdline::Spotify to access it from the YAML file:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">OAuth::Cmdline::Spotify</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$oauth</span> <span class="operator">=</span> <span class="word">OAuth::Cmdline::Spotify</span><span class="operator">-></span><span class="word">new</span><span class="structure">();</span><br /><span class="keyword">my</span> <span class="symbol">$token</span> <span class="operator">=</span> <span class="symbol">$oauth</span><span class="operator">-></span><span class="word">access_token</span><span class="structure">;</span></code></pre>
<h3 id="Accessing-the-Spotify-Web-API">Accessing the Spotify Web API</h3>
<p>Now I've got the complicated authentication and authorization out of the way it's now just a simple matter of programming to complete my interface to Spotify:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">LWP::UserAgent</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">HTTP::Request::Common</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$ua</span> <span class="operator">=</span> <span class="word">LWP::UserAgent</span><span class="operator">-></span><span class="word">new</span><span class="structure">();</span><br /><span class="keyword">my</span> <span class="symbol">$BASE_URL</span> <span class="operator">=</span> <span class="single">'https://api.spotify.com'</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">@STANDARD_HEADERS</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="single">'Accept'</span> <span class="operator">=></span> <span class="single">'application/json'</span><span class="operator">,</span><br /> <span class="single">'Authorization'</span> <span class="operator">=></span> <span class="double">"Bearer $token"</span><span class="operator">,</span><br /> <span class="single">'Content-Type'</span> <span class="operator">=></span> <span class="single">'application/json'</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="keyword">sub</span> <span class="word">get</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$path</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$url</span> <span class="operator">=</span> <span class="word">URI</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><span class="double">"$BASE_URL$path"</span><span class="structure">);</span><br /> <span class="symbol">$url</span><span class="operator">-></span><span class="word">query_form</span><span class="structure">(</span> <span class="magic">@_</span> <span class="structure">);</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$response</span> <span class="operator">=</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">request</span><span class="structure">(</span><br /> <span class="word">GET</span> <span class="symbol">$url</span><span class="operator">,</span><br /> <span class="symbol">@STANDARD_HEADERS</span><span class="operator">,</span><br /> <span class="structure">);</span><br /> <span class="word">die</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">message</span> <span class="word">unless</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">is_success</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">decode_json</span><span class="structure">(</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">content</span> <span class="structure">);</span><br /><span class="structure">}</span><br /><br /><span class="keyword">sub</span> <span class="word">post</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$path</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$data</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$url</span> <span class="operator">=</span> <span class="word">URI</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><span class="double">"$BASE_URL$path"</span><span class="structure">);</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$response</span> <span class="operator">=</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">request</span><span class="structure">(</span><br /> <span class="word">POST</span> <span class="symbol">$url</span><span class="operator">,</span><br /> <span class="symbol">@STANDARD_HEADERS</span><span class="operator">,</span><br /> <span class="word">Content</span> <span class="operator">=></span> <span class="word">encode_json</span><span class="structure">(</span> <span class="symbol">$data</span> <span class="structure">)</span><span class="operator">,</span><br /> <span class="structure">);</span><br /> <span class="word">die</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">message</span> <span class="word">unless</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">is_success</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">decode_json</span><span class="structure">(</span> <span class="symbol">$response</span><span class="operator">-></span><span class="word">content</span> <span class="structure">);</span><br /><span class="structure">}</span></code></pre>
<p>And finally I can write the code to build my playlist:</p>
<pre><code class="code-listing"><span class="comment"># find the uri of the best matching song for the<br /># track name and artist<br /></span><span class="keyword">my</span> <span class="symbol">@found_tracks</span><span class="structure">;</span><br /><span class="keyword">foreach</span> <span class="keyword">my</span> <span class="symbol">$itunes_track</span> <span class="structure">(</span><span class="cast">@</span><span class="structure">{</span> <span class="symbol">$itunes_tracks</span> <span class="structure">})</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$title</span> <span class="operator">=</span> <span class="symbol">$itunes_track</span><span class="operator">-></span><span class="structure">{</span><span class="word">title</span><span class="structure">};</span><br /> <span class="symbol">$title</span> <span class="operator">=~</span> <span class="substitute">s/[(][^)]+[)]//g</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$artist</span> <span class="operator">=</span> <span class="symbol">$itunes_track</span><span class="operator">-></span><span class="structure">{</span><span class="word">artist</span><span class="structure">};</span><br /> <span class="symbol">$artist</span> <span class="operator">=~</span> <span class="substitute">s/,.*//g</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$search_string</span> <span class="operator">=</span> <span class="double">"$title $artist"</span><span class="structure">;</span><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"Matching $search_string\n"</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$search</span> <span class="operator">=</span> <span class="word">get</span><span class="structure">(</span><br /> <span class="single">'/v1/search'</span><span class="operator">,</span><br /> <span class="word">q</span> <span class="operator">=></span> <span class="symbol">$search_string</span><span class="operator">,</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'track'</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$spotify_track</span> <span class="operator">=</span> <span class="symbol">$search</span><span class="operator">-></span><span class="structure">{</span><span class="word">tracks</span><span class="structure">}{</span><span class="word">items</span><span class="structure">}[</span><span class="number">0</span><span class="structure">];</span><br /> <span class="keyword">unless</span> <span class="structure">(</span><span class="symbol">$spotify_track</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"No match!\n"</span><span class="structure">;</span><br /> <span class="word">next</span><span class="structure">;</span><br /> <span class="structure">}</span><br /> <span class="word">push</span> <span class="symbol">@found_tracks</span><span class="operator">,</span> <span class="symbol">$spotify_track</span><span class="operator">-></span><span class="structure">{</span><span class="word">uri</span><span class="structure">};</span><br /><span class="structure">}</span><br /><br /><span class="comment"># create a new playlist<br /></span><span class="keyword">my</span> <span class="symbol">$playlist_id</span> <span class="operator">=</span> <span class="word">post</span><span class="structure">(</span><br /> <span class="single">'/v1/users/2shortplanks/playlists'</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">name</span> <span class="operator">=></span> <span class="literal">q{Mark's Christmas Playlist}</span> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">)</span><span class="operator">-></span><span class="structure">{</span><span class="word">id</span><span class="structure">};</span><br /><br /><span class="comment"># add the tracks to that playlist<br /></span><span class="word">post</span><span class="structure">(</span><br /> <span class="double">"/v1/playlists/$playlist_id/tracks"</span><span class="operator">,</span><br /> <span class="structure">{</span> <span class="word">uris</span> <span class="operator">=></span> <span class="cast">\</span><span class="symbol">@found_tracks</span> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">);</span></code></pre>
<h3 id="And-done">And done</h3>
<p>Running the script produces a best-effort version of the iTunes playlist <a href="https://open.spotify.com/playlist/5hyCf0PuWFknHdmGNjarFZ">on Spotify</a> (hampered not in the least by the fact that Slade doesn't distribute via Spotify).</p>
<p><center><iframe src="https://open.spotify.com/embed/playlist/5hyCf0PuWFknHdmGNjarFZ" width="600" height="380" frameborder="0" allowtransparency="true" allow="encrypted-media"></iframe></center>
</p>
<p>Happy listening!</p>
</div>2018-12-17T00:00:00ZMark FowlerHandling complex data structureshttp://perladvent.org/2018/2018-12-16.html<div class='pod'><p>One of the challenging aspects of the Wise Old Elf's tasks is dealing with all of the various lists of presents requested by all of the children. These arrive in a variety of formats with different structures and these need to be all displayed on a webpage so each can be checked and validated.</p>
<p>This causes a problem displaying unstructured data on a web page.</p>
<p>Luckily there is a module which helps here - the <a href="https://metacpan.org/module/Hash::Flatten">Hash::Flatten</a> module takes an arbitrary data structure and converts it into a simple hash with the key to each element being a Template Toolkit style token refering to the element in the data structure.</p>
<p>For example, given a deep data structure such as this:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$data</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'England'</span> <span class="operator">=></span> <span class="structure">{</span> <br /> <span class="single">'Bucks'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Milton Keynes'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Some Road'</span> <span class="operator">=></span> <span class="structure">{</span> <br /> <span class="number">4</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Bloggs'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Fred'</span> <span class="operator">=></span> <span class="single">'Fire Engine'</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'USA'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Maine'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'York'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Main Road'</span> <span class="operator">=></span> <span class="structure">{</span> <br /> <span class="single">'White House'</span> <span class="operator">=></span> <span class="structure">{</span> <br /> <span class="single">'Naughty Child'</span> <span class="operator">=></span> <span class="single">'Ipad'</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">};</span></code></pre>
<p>This would be awkward to display on a web page but using the <code>flatten</code> function of Hash::Flatten you can convert it into a simple flat hash which can then be displayed as a simple table. i.e.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Hash::Flatten</span> <span class="words">qw/ flatten /</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$flathash</span> <span class="operator">=</span> <span class="word">flatten</span><span class="structure">(</span> <span class="symbol">$data</span> <span class="structure">);</span></code></pre>
<p>The variable <code>$flathash</code> will then contain:</p>
<pre><code class="code-listing"><span class="symbol">$flathash</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'England.Bucks.Milton Keynes.Some Road.4.Bloggs.Fred'</span> <span class="operator">=></span> <span class="single">'Fire Engine'</span><span class="operator">,</span><br /> <span class="single">'USA.Maine.York.Main Road.White House.Naughty Child'</span> <span class="operator">=></span> <span class="single">'Ipad'</span><span class="operator">,</span><br /><span class="structure">};</span></code></pre>
<p>You can now easily present this in a table:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">HTML::Entities</span> <span class="words">qw(encode_entities)</span><span class="structure">;</span><br /><span class="word">print</span> <span class="double">"<table>"</span><span class="structure">;</span><br /><span class="keyword">while</span> <span class="structure">(</span><span class="keyword">my</span> <span class="symbol">@kv</span> <span class="operator">=</span> <span class="word">each</span> <span class="symbol">$flathash</span><span class="operator">-></span><span class="cast">%*</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"<tr>"</span><span class="structure">;</span><br /> <span class="word">print</span> <span class="double">"<td>"</span><span class="operator">,</span> <span class="word">encode_entities</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">)</span><span class="operator">,</span> <span class="double">"</td>"</span><br /> <span class="word">for</span> <span class="symbol">@kv</span><span class="structure">;</span><br /> <span class="word">print</span> <span class="double">"</tr>"</span><span class="structure">;</span><br /><span class="structure">}</span><br /><span class="word">print</span> <span class="double">"</table>"</span><span class="structure">;</span></code></pre>
<table border="1">
<tr>
<td>England.Bucks.Milton Keynes.Some Road.4.Bloggs.Fred</td>
<td>Fire Engine</td>
</tr>
<tr>
<td>USA.Maine.York.Main Road.White House.Naughty Child</td>
<td>Ipad</td>
</tr>
</table>
<p>Or store it in a flat DBM File or process it in any number of ways! You can even go the other way too by using the <b>unflatten</b> function to turn the flattened data structure back into a nested data structure:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Hash::Flatten</span> <span class="words">qw( unflatten )</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$data</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'food.reindeer'</span> <span class="operator">=></span> <span class="single">'carrots'</span><span class="operator">,</span><br /> <span class="single">'food.santa.pie'</span> <span class="operator">=></span> <span class="single">'mince'</span><span class="operator">,</span><br /> <span class="single">'food.santa.drink'</span> <span class="operator">=></span> <span class="single">'sherry'</span><span class="operator">,</span><br /><span class="structure">};</span><br /><br /><span class="keyword">use</span> <span class="word">Data::Dumper</span><span class="structure">;</span><br /><span class="word">print</span> <span class="word">Dumper</span> <span class="word">unflatten</span><span class="structure">(</span><span class="symbol">$data</span><span class="structure">);</span></code></pre>
<p>Which prints out:</p>
<pre><code class="code-listing"><span class="symbol">$VAR1</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'food'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'reindeer'</span> <span class="operator">=></span> <span class="single">'carrots'</span><span class="operator">,</span><br /> <span class="single">'santa'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'pie'</span> <span class="operator">=></span> <span class="single">'mince'</span><span class="operator">,</span><br /> <span class="single">'drink'</span> <span class="operator">=></span> <span class="single">'sherry'</span><br /> <span class="structure">}</span><br /> <span class="structure">}</span><br /><span class="structure">};</span></code></pre>
<p>The elves actually use Hash::Flatten for a lot of things in various contexts. Through passing a second argument (or using the OO-interface) it's possible to control what deliminator are used to join / split the parts of hash and arrays.</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$data</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'Beano'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Denis'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'Catapult'</span><span class="operator">,</span><span class="single">'Dog Treats'</span><span class="structure">]</span><span class="operator">,</span><br /> <span class="single">'Minnie'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'Tricks'</span><span class="operator">,</span> <span class="single">'Hair Bows'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="single">'Roger'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'Gadgets'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'Archie'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'Josie'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'Guitar'</span><span class="operator">,</span> <span class="single">'Cat Ears'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="single">'Sabrina'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'Spellbook'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">};</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$flathash</span> <span class="operator">=</span> <span class="word">flatten</span><span class="structure">(</span> <span class="symbol">$data</span><span class="operator">,</span> <span class="structure">{</span><br /> <span class="word">HashDelimiter</span> <span class="operator">=></span> <span class="single">' -> '</span><span class="operator">,</span> <br /> <span class="word">ArrayDelimiter</span> <span class="operator">=></span> <span class="single">', '</span><span class="operator">,</span><br /> <span class="structure">});</span></code></pre>
<p>And now the flattened hash looks like this:</p>
<pre><code class="code-listing"><span class="symbol">$flathash</span> <span class="operator">=</span> <span class="structure">{</span><br /> <span class="single">'Beano -> Denis'</span> <span class="operator">=></span> <span class="single">'Catapult, Dog Treats'</span><span class="operator">,</span><br /> <span class="single">'Beano -> Minnie'</span> <span class="operator">=></span> <span class="single">'Tricks, Hair Bows'</span><span class="operator">,</span><br /> <span class="single">'Beano -> Rodger'</span> <span class="operator">=></span> <span class="single">'Gadgets'</span><span class="operator">,</span><br /> <span class="single">'Archie -> Josie'</span> <span class="operator">=></span> <span class="single">'Guitar, Cat Ears'</span><span class="operator">,</span><br /> <span class="single">'Archie -> Sabrina'</span> <span class="operator">=></span> <span class="single">'Spellbook'</span><span class="operator">,</span><br /><span class="structure">}</span></code></pre>
<p>Happy data munging!</p>
</div>2018-12-16T00:00:00ZTony EdwardsonSpot The Differencehttp://perladvent.org/2018/2018-12-15.html<div class='pod'><p>"Sometimes", Binky the Elf thought to himself, "you end up making life difficult for yourself".</p>
<p>Sure, implementing the new low-bandwidth communications protocol to Santa's sleigh had been Binky's best idea yet, but, of course, someone had to support the system when it went wrong. And that someone? The Elf whose bright idea it was in the first place.</p>
<p>The problem that Binky's team was solving was that Santa's crew had needed to constantly update the status of present delivery back to the North Pole, but the sleigh kept entering communication dead zones where they couldn't make use of the HTTP endpoints they'd originally setup. Binky has addressed this problem by encoding each present delivery status as a single character, and sending a constant stream of present updates back to home base. Rather than two hundred HTTP calls, the stream just sent back the following highly compact data:</p>
<pre><code> tweiadzjxiwxbbqrhlrrjcprgmctbtusbvwkvxwu
afnysopgavusxkcejwbsbxajoncdkiiwzbkhjzem
ijcqrbnkiynhpvnyazgejwbtzkcnagtnjcqxvrwm
qcxdvczunrrkcusgmzhsglznliheqivqhkhavvas
urwybcffdnqkqwnqdblzmgppolxhbkkhktjqqozp</code></pre>
<p>Highly compact data, however, that Binky was finding out to be really hard to debug. On one of the sleigh's many dry runs it had returned the wrong data <i>somewhere</i> in the stream of less-than-readable letters, and Binky had to track each of them down.</p>
<p>Binky stared in his terminal at the files <code>a.txt</code> and <code>b.txt</code> that represented the expected and actual output for the first of the several hundred delivery sites with corrupt data.</p>
<p>Finding the line with the different character was simple - that's what <code>diff</code> is for:</p>
<pre><code> shell$ diff -u a.txt b.txt
--- a.txt 2018-12-05 05:59:11.000000000 -0500
+++ b.txt 2018-12-05 05:59:26.000000000 -0500
@@ -1,5 +1,5 @@
tweiadzjxiwxbbqrhlrrjcprgmctbtusbvwkvxwu
afnysopgavusxkcejwbsbxajoncdkiiwzbkhjzem
-ijcqrbnkiynhpvnyazgejwbtzkcnagtnjcqxvrwm
+ijcqrbnkiynhpvnyazgejwdtzkcnagtnjcqxvrwm
qcxdvczunrrkcusgmzhsglznliheqivqhkhavvas
urwybcffdnqkqwnqdblzmgppolxhbkkhktjqqozp</code></pre>
<p>But finding the actual differing character on that line that...that was much harder.</p>
<p>Binky tried moving character by character with his finger on the screen comparing each line with the one above it, but by the time he got half way through his had eyes glazed over. And there wasn't just one of these files...there were hundreds to go! Either he was going to have to prop his eyes open with candy canes or he was going to have to be smarter.</p>
<p>It was time for Binky to use one of the virtues of a Perl programmer: Laziness! Time to search the CPAN for a solution.</p>
<h3 id="App::ccdiff">App::ccdiff</h3>
<p>One of the humans, H.Merijn Brand, had had a similar problem when debugging terminal capabilities files (which also tend to be somewhat unreadable). In the end he'd written a diff utility <code>ccdiff</code> that could not only identify what <i>line</i> had changed in the file, but could also indicate what <i>characters</i> had changed on the line.</p>
<center><img src="ccdiff1.jpg" width="528" height="360" alt="ccdiff screenshot"></center>
<p>Binky was over the moon. With this tool he'd be able to spot the changes right away.</p>
<p><code>ccdiff</code> is quite configurable and has some handy extra features. You can configure the color scheme for the various different characters. Very handy if you're even slightly red-green colorblind (like the author of this article.)</p>
<center><img src="ccdiff2.jpg" width="528" height="360" alt="ccdiff screenshot"></center>
<p><code>ccdiff</code> even has settings suitable for generating output that's easy to copy and paste into an email, bug tracker, or other tool that'll lose the color information</p>
<pre><code> shell$ ccdiff --ascii --no-color -m -u a.txt b.txt
--- a.txt Wed Dec 5 05:59:11 2018
+++ b.txt Wed Dec 5 05:59:26 2018
3,3c3,3
tweiadzjxiwxbbqrhlrrjcprgmctbtusbvwkvxwu
afnysopgavusxkcejwbsbxajoncdkiiwzbkhjzem
-ijcqrbnkiynhpvnyazgejwbtzkcnagtnjcqxvrwm
- ^
+ijcqrbnkiynhpvnyazgejwdtzkcnagtnjcqxvrwm
+ ^
qcxdvczunrrkcusgmzhsglznliheqivqhkhavvas
urwybcffdnqkqwnqdblzmgppolxhbkkhktjqqozp</code></pre>
<p>Awesome! Now Binky was able to open a github issue with the problem identified. Unfortunatly, some poor elf had to actually <i>fix</i> that bug, and it looked like it might be another long night for Binky...</p>
</div>2018-12-15T00:00:00ZMark FowlerThe future is nowhttp://perladvent.org/2018/2018-12-14.html<div class='pod'><p>Today was a big day for Gnomo Knullpointer. They had recently started a new position as a coding elf at Santa's workshop, and they were eager to fulfil their calling as Santa's helper, and to start actually <i>helping Santa</i>. And having finally passed their probation, today they were getting their first <i>real</i> assignment. This was all <i>very exciting</i>.</p>
<p>"So you see, Gnomo... is that how you pronounce your name?" asked Santa, without really expecting a reply. "We need to modernise the way we send our gifts, and I want you to write a ... package, is it? One of those things to do it. Think you're up to the task?".</p>
<p>Gnomo was very much up to the task.</p>
<p>Always eager to please, Gnomo set out to write what was needed. The first version looked a bit like this:</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">Santa::Helper</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">HTTP::Tiny</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$ua</span> <span class="operator">=</span> <span class="word">HTTP::Tiny</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">send_gifts</span> <span class="structure">{</span><br /> <span class="keyword">while</span> <span class="structure">(</span><span class="magic">@_</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span> <span class="symbol">$address</span><span class="operator">,</span> <span class="symbol">$gift</span> <span class="structure">)</span> <span class="operator">=</span> <span class="structure">(</span> <span class="core">shift</span><span class="operator">,</span> <span class="core">shift</span> <span class="structure">);</span><br /> <span class="keyword">my</span> <span class="symbol">$res</span> <span class="operator">=</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">post</span><span class="structure">(</span> <span class="symbol">$address</span><span class="operator">,</span> <span class="symbol">$gift</span> <span class="structure">);</span><br /> <span class="keyword">if</span> <span class="structure">(</span><span class="symbol">$res</span><span class="operator">-></span><span class="structure">{</span><span class="word">success</span><span class="structure">})</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Gift was delivered to $address!\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><br /> <span class="keyword">else</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Could not deliver gift to $address :(\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><br /> <span class="structure">}</span><br /><span class="structure">}</span></code></pre>
<p>Which would then be called without much flourish like so:</p>
<pre><code class="code-listing"><span class="word">Santa::Helper::send_gifts</span><span class="structure">(</span><br /> <span class="single">'http://httpbin.org/delay/5'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'teddy_bear'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/status/500'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'raspberrypi'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/delay/1'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'puppy'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/delay/2'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'bycicle'</span> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">)</span></code></pre>
<p>"I like it!" said Santa, impressed by the colours of the syntax highlighter. "What do you think, Ada?", he said, looking at Ada Slashdóttir the senior developer in the team.</p>
<p>"Well, yeah, it's all well and good. It certainly gets the job done. But it's not very <i>efficient</i>", said Ada. "This delivers the gifts one after the other. We have a lot of helpers. Why not send all the gifts out at the same time, and then let them take as long as they must? When the gifts are all delivered, then we can send the sleigh back to fetch more."</p>
<p>"Oh, you mean send the gifts <i>asynchronously</i>", replied Gnomo.</p>
<p>Santa, was starting to lose focus. He had never been into the implementation details, and preferred to surround himself with people he could trust, like Ada. He let the developer elves flex their intellectual muscles, confident they could sort it out.</p>
<p>After a little more coding, Gnomo came back with this:</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">Santa::Helper</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">IO::Async::Loop</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Net::Async::HTTP</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">URI</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$ua</span> <span class="operator">=</span> <span class="word">Net::Async::HTTP</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><br /> <span class="word">max_connections_per_host</span> <span class="operator">=></span> <span class="number">0</span><span class="operator">,</span><br /> <span class="word">fail_on_error</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="word">IO::Async::Loop</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">add</span><span class="structure">(</span> <span class="symbol">$ua</span> <span class="structure">);</span><br /><br /><span class="keyword">sub</span> <span class="word">send_gifts</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">@requests</span><span class="structure">;</span><br /> <span class="keyword">while</span> <span class="structure">(</span><span class="magic">@_</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span> <span class="symbol">$address</span><span class="operator">,</span> <span class="symbol">$gift</span> <span class="structure">)</span> <span class="operator">=</span> <span class="structure">(</span> <span class="core">shift</span><span class="operator">,</span> <span class="core">shift</span> <span class="structure">);</span><br /> <span class="word">push</span> <span class="symbol">@requests</span><span class="operator">,</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">POST</span><span class="structure">(</span><br /> <span class="word">URI</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="symbol">$address</span> <span class="structure">)</span> <span class="operator">=></span> <span class="symbol">$gift</span><span class="operator">,</span><br /> <span class="word">on_response</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Gift was delivered to $address!\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="word">on_error</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Could not deliver gift to $address :(\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">);</span><br /> <span class="structure">}</span><br /> <span class="symbol">@requests</span><span class="structure">;</span><br /><span class="structure">}</span></code></pre>
<p>Ada was pleased. "That's more like it!"</p>
<p>"But hold on a second", said Duende Juniorsson, the junior developer. "If I call this like we did with the previous one, <i>nothing happens</i>."</p>
<p>Santa looked worried.</p>
<p>"Oh, it only <i>looks</i> like nothing happens because the function now returns immediately with a Future, and you are not waiting for it to actually get any work done", explained Gnomo.</p>
<p>"Exactly", said Ada. "A Future represents the result of an action that, like its name implies, will be completed in the future. So you need to wait for it to complete before you can get a result."</p>
<p>Santa looked worried <i>and</i> confused.</p>
<p>"So how <i>should</i> we call it, then?", asked Duende.</p>
<p>Gnomo replied: "You can wait on an individual Future, like this":</p>
<pre><code class="code-listing"><span class="word">Santa::Helper::send_gifts</span><span class="structure">(</span><br /> <span class="single">'http://httpbin.org/delay/5'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'teddy_bear'</span> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">)</span><span class="operator">-></span><span class="word">get</span><span class="structure">;</span></code></pre>
<p>"Or use a convergent Future, to wait for a set of them at once", suggested Ada:</p>
<pre><code class="code-listing"><span class="word">Future</span><span class="operator">-></span><span class="word">wait_all</span><span class="structure">(</span><br /> <span class="word">Santa::Helper::send_gifts</span><span class="structure">(</span><br /> <span class="single">'http://httpbin.org/delay/5'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'teddy_bear'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/status/500'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'raspberrypi'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/delay/1'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'puppy'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'http://httpbin.org/delay/2'</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">code</span> <span class="operator">=></span> <span class="single">'bycicle'</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">)</span><br /><span class="structure">)</span><span class="operator">-></span><span class="word">get</span><span class="structure">;</span></code></pre>
<p>Santa started sweating profusely. Convergent <i>what</i>? "I, um... just want it to send a gift... I didn't think we were going to start messing with the timeline, at least not just yet...". The developer elves looked at each other.</p>
<p>"I mean, I like what Ada wanted, to make all the gifts be delivered asymptotically...", said Santa.</p>
<p>"Asynchronously", corrected Ada.</p>
<p>"Right", said Santa. "Asynchronously, that's what I said. I liked that bit. But I think things got a little confusing."</p>
<p>Duende agreed. "Yeah, I mean, I'm only getting started, but even I could clearly understand what <code>send_gift(...)</code> meant. But I've been staring at <code>send_gift(...)->get</code> for a bit, and I still have no idea what it does. And don't even get me started on having ... futures that wait on other futures?".</p>
<p>"Yes, this is true", said Ada. "The interface has got a bit more complicated, and not every user of this helper will want to deal directly with the Futures, or even know or care what they are."</p>
<p>"How about," suggested Gnomo, "we leave the non-blocking implementation in the background, exposed through a function for those who need it, but make another function that can be used as a blocking interface for those who just want to send the gifts already?" This is what they had in mind:</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">Santa::Helper</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">IO::Async::Loop</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Net::Async::HTTP</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">URI</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$ua</span> <span class="operator">=</span> <span class="word">Net::Async::HTTP</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><br /> <span class="word">max_connections_per_host</span> <span class="operator">=></span> <span class="number">0</span><span class="operator">,</span><br /> <span class="word">fail_on_error</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="word">IO::Async::Loop</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">add</span><span class="structure">(</span> <span class="symbol">$ua</span> <span class="structure">);</span><br /><br /><span class="comment"># Non-blocking, returns Futures<br /></span><span class="keyword">sub</span> <span class="word">send_gifts_f</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">@requests</span><span class="structure">;</span><br /> <span class="keyword">while</span> <span class="structure">(</span><span class="magic">@_</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span> <span class="symbol">$address</span><span class="operator">,</span> <span class="symbol">$gift</span> <span class="structure">)</span> <span class="operator">=</span> <span class="structure">(</span> <span class="core">shift</span><span class="operator">,</span> <span class="core">shift</span> <span class="structure">);</span><br /> <span class="word">push</span> <span class="symbol">@requests</span><span class="operator">,</span> <span class="symbol">$ua</span><span class="operator">-></span><span class="word">POST</span><span class="structure">(</span><br /> <span class="word">URI</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="symbol">$address</span> <span class="structure">)</span> <span class="operator">=></span> <span class="symbol">$gift</span><span class="operator">,</span><br /> <span class="word">on_response</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Gift was delivered to $address!\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="word">on_error</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="word">print</span> <span class="double">"Could not deliver gift to $address :(\n"</span><span class="structure">;</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">);</span><br /> <span class="structure">}</span><br /> <span class="symbol">@requests</span><span class="structure">;</span><br /><span class="structure">}</span><br /><br /><span class="comment"># Blocking, waits for all the Futures before returning<br /></span><span class="keyword">sub</span> <span class="word">send_gifts</span> <span class="structure">{</span> <span class="word">Future</span><span class="operator">-></span><span class="word">wait_all</span><span class="structure">(</span> <span class="word">send_gifts_f</span><span class="structure">(</span> <span class="magic">@_</span> <span class="structure">)</span> <span class="structure">)</span><span class="operator">-></span><span class="word">get</span> <span class="structure">}</span></code></pre>
<p>"So now if you want or care about the Futures," explained Gnomo, "you can call the functions that end with <code>_f</code>, and know what those will return immediately with a Future. But you can also use the helper as a vanilla non-blocking library if you want".</p>
<p>"Incidentally," commented Ada, "there's already a bunch of libraries on CPAN that do exactly this. Some of them use <code>_p</code> as the suffix, and others use entirely different names when they make sense. What matters is that Futures allow for this sort of versatility, so even non-blocking libraries, or those based on them, can also provide blocking interfaces."</p>
<p>Santa nodded, wondering if he was expected to say something. It looked like he was safe, though, since everyone was smiling. He decided to leave the details in their hands and go back to more pressing matters.</p>
<p>After all, those milk and cookies weren't going to eat themselves!</p>
</div>2018-12-14T00:00:00ZJosé Joaquín AtriaLost in legacy argumentshttp://perladvent.org/2018/2018-12-13.html<div class='pod'><h2 id="Working-with-legacy-code">Working with legacy code</h2>
<p>Delivering Christmas gifts is a hard job. Every culture and every country has it's own different traditions. Each country has it's own different gift bringer - an old man (Santa Claus, Father Christmas, Grandfather Frost, ...), a child (Baby Jesus, Christ Child, Child God, ...) or other (The Three Kings, Yule Goat, ...) And each country has different days, from Christmas Eve, or January 7th following the Gregorian calendar, or even January 2nd in Japan.</p>
<p>All these entities worked together in the Society of Annual Noel Transfer Association (S.A.N.T.A) to coordinate their deliveries. And it was someone's job to keep track of what everyone had to do.</p>
<p>To help with this problem they'd started with a <code>deliver_gift</code> subroutine. Well intention, well named, but over time as more members had joined S.A.N.T.A. it'd grown little bit out of control:</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">deliver_gift</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$gift</span><span class="operator">,</span> <span class="symbol">$child</span><span class="operator">,</span> <span class="symbol">$location</span><span class="operator">,</span> <span class="symbol">$delivery_day</span><span class="operator">,</span> <span class="symbol">$old_man_bringer</span><span class="operator">,</span> <span class="symbol">$delivery_method</span><span class="operator">,</span> <span class="symbol">$child_bringer</span><span class="operator">,</span> <span class="symbol">$other_bringer</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /><span class="comment"> # few hundreds of lines of code untouched this millennium follows<br /></span> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<h2 id="Nightmare-Before-Christmas">Nightmare Before Christmas</h2>
<p>The Nightmare begins for our maintainer - this function needs yet more changes. He needs to add a <code>delivery_time</code> argument to account for the time of day when presents are delivered (which again, apparently, varies between country).</p>
<p>Now:</p>
<ul>
<li>only first three arguments are mandatory</li>
<li>function is called from many places</li>
<li>optional arguments are often omitted</li>
</ul>
<h3 id="A-Bad-Solution">A Bad Solution</h3>
<p>The hacky solution that maintainer can implement is to put a new argument after last mandatory field. Pity our poor maintainer who chooses this solution and failed to update every last place that the function is called in the production code and accidentally calls the function with an argument in the wrong place!</p>
<h3 id="A-Slightly-Better-Solution">A Slightly Better Solution</h3>
<p>Our poor maintainer wants to spend Christmas at home with his family, not on call for bad code changes! At this point his slightly better solution is simple: Quit his job and enjoy peace and harmony on earth!</p>
<h3 id="The-Better-Solution">The Better Solution</h3>
<p>It was time for our maintainer to bite the bullet and change the function's calling style from positional to named arguments so it can be called like so:</p>
<pre><code class="code-listing"><span class="word">deliver_gift</span> <span class="structure">(</span><br /> <span class="word">gift</span> <span class="operator">=></span> <span class="symbol">$gift</span><span class="operator">,</span><br /> <span class="word">child</span> <span class="operator">=></span> <span class="symbol">$child</span><span class="operator">,</span><br /> <span class="word">location</span> <span class="operator">=></span> <span class="symbol">$location</span><span class="operator">,</span><br /> <span class="operator">...,</span><br /><span class="comment"> # our new functionality<br /></span> <span class="word">delivery_time</span> <span class="operator">=></span> <span class="single">'after-dinner'</span><span class="operator">,</span><br /> <span class="structure">);</span></code></pre>
<p>But how to do that without breaking all the existing code?</p>
<h4 id="Plan-A:-deliver_gift_with_named_arguments">Plan A: deliver_gift_with_named_arguments</h4>
<p>Our maintainer's first plan was a multi-step process:</p>
<ul>
<li>implementation: introduce a new function</li>
<li>implementation: make the old function wrapper over the new one</li>
<li>implementation: replace the old function's calls with new one</li>
<li>cleanup: rename the new function into the old name, create a wrapper</li>
<li>cleanup: replace the new function name with the old name</li>
<li>cleanup: finally get rid of the wrapper</li>
</ul>
<p>Oh My! So many steps!</p>
<p>In these days of zero downtime deployment via microservices a lot of synchronization points are required for such a complicated plan, multiplied by number of modules that uses this function.</p>
<p>Could we get all this done by Christmas? Unlikely! Our maintainer needed a better plan!</p>
<h4 id="Plan-B:-Make-function-to-accept-both-positional-and-named-arguments">Plan B: Make function to accept both positional and named arguments</h4>
<p>There is a handy module on CPAN called <a href="https://metacpan.org/module/Sub::Params">Sub::Params</a> that handles calling style duality.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Sub::Params</span> <span class="words">qw[ named_or_positional_arguments ]</span><span class="structure">;</span><br /><span class="keyword">sub</span> <span class="word">delivery_gift</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$self</span><span class="operator">,</span> <span class="symbol">@args</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">%args</span> <span class="operator">=</span> <span class="word">named_or_positional_arguments</span> <span class="structure">(</span><br /> <span class="word">args</span> <span class="operator">=></span> <span class="cast">\</span><span class="symbol">@args</span><span class="operator">,</span><br /> <span class="word">names</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="single">'gift'</span><span class="operator">,</span><br /> <span class="single">'child'</span><span class="operator">,</span><br /> <span class="single">'location'</span><span class="operator">,</span><br /> <span class="single">'delivery_day'</span><span class="operator">,</span><br /> <span class="single">'old_man_bringer'</span><span class="operator">,</span><br /> <span class="single">'delivery_method'</span><span class="operator">,</span><br /> <span class="single">'child_bringer'</span><span class="operator">,</span><br /> <span class="single">'other_bringer'</span><span class="operator">,</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><span class="structure">}</span></code></pre>
<p>With these changes the function can be called in either style:</p>
<pre><code class="code-listing"><span class="word">delivery_gift</span> <span class="structure">(</span><br /> <span class="symbol">$gift</span><span class="operator">,</span><br /> <span class="symbol">$child</span><span class="operator">,</span><br /> <span class="operator">...</span><br /><span class="structure">);</span><br /><br /><span class="word">delivery_gift</span> <span class="structure">(</span><br /> <span class="word">gift</span> <span class="operator">=></span> <span class="symbol">$gift</span><span class="operator">,</span><br /> <span class="word">child</span> <span class="operator">=></span> <span class="symbol">$child</span><span class="operator">,</span><br /> <span class="operator">...</span><br /><span class="structure">);</span></code></pre>
<p>Reducing the number of necessary changes into merely:</p>
<ul>
<li>implementation: introduce duality</li>
<li>implementation: change positional style to named style</li>
<li>cleanup: remove duality</li>
</ul>
<p>Merry Christmas and Happy Refactoring :-)</p>
<h3 id="Thanks">Thanks</h3>
<p><a href="https://gooddata.com/">GoodData</a> for contributing to open source.</p>
</div>2018-12-13T00:00:00ZBranislav ZahradníkGrowing Christmas Treeshttp://perladvent.org/2018/2018-12-12.html<div class='pod'><p>Twinkly the elf was in a bit of a panic. She had been given the job of producing a series of designs for this year's North Pole Christmas cards. So far she had made no progress. And the print deadline was approaching fast.</p>
<p>The problem was just too complex. It wasn't just a case of taking a mildly embarrassing photo of the Clauses and then printing that on hundreds of cards. No, Santa had decided that he didn't want a photo this year. He wanted cute graphics of Christmas trees. Also (and this was the hard part), he wanted dozens of different designs with different numbers of tree and tree of many different sizes. There just wasn't time to create that many designs.</p>
<p>Twinkly was having coffee with the Wise Old Elf in the North Pole Canteen when she mentioned her problem to him. He suggested that she used SVG to produce the images. Twinkly had never heard of SVG, so she did some research.</p>
<h3 id="Two-Types-of-Graphics">Two Types of Graphics</h3>
<p>Most image files that we come across every day are <b>raster</b> images. Basically, the image is made up of a two-dimensional grid of points (called <b>picture elements</b> or <b>pixels</b>). Each pixel is set to be a particular colour and when we look at the complete set of coloured pixels, we see it as an image.</p>
<p>Raster images are great for many uses. All of the JPG and PNG and GIF images that you come across every say on the internet are raster images. But they have a couple of downsides - one of which is that they don't scale very well. When you view a raster image at the intended size, you don't see the inidividual pixels. But as you scale up the image, there will come a point where each pixel becomes large enough to be seen. Initially, you might notice that a straight edge starts to look a bit jagged. Eventually, you'll see every pixel and the image turns into an unrecognisable pile of squares.</p>
<p>Vector images are different. Vector images don't contain instructions saying that "this pixel is red" or "that pixel is blue". They contain higher level instructions like "draw a red circle with this radius, centred on this point". And as you scale up the drawing area, those instructions remain just as valid. You'll get a bigger circle, but one that is still drawn with sharp edges.</p>
<p>SVG (for "Scalable Vector Graphics") is an XML format for describing vector graphics. It has become very popular on the web and many browsers now have build-in support for displaying SVG images.</p>
<p>Not all images are suitable for being represented in the SVG format. As an SVG image is made up of lots (possibly hundreds) of shapes, something like a photo doesn't really work in this format. But if you can describe your image in terms of the shapes that make it up, then it would be a good candidate for being turned into an SVG document.</p>
<p>For example, a cartoon representation of a Christmas tree.</p>
<h3 id="Varying-the-Design">Varying the Design</h3>
<p>But that only gets us part of the way there. Twinkly needs to produce many different designs of her Christmas tree. How does SVG help us there?</p>
<p>An SVG document isn't so much a description of an image. It's more of a recipe that tells you how to produce the image. And when you have a recipe, it's easy to vary parts of the recipe in order to vary the results we get out at the end. So we need to write a program which generates various different Christmas trees, depending on its input parameters. And, luckily, there is a CPAN module called SVG which we can use to output SVG documents.</p>
<p>Let's think a little about what we need to draw a Christmas tree. I'm planning on something like this:</p>
<center><img src="shapes.jpg" width="332" height="500" border="1"></center>
<p>The main tree is four triangles. There's a brown rectangle at the bottom for the trunk and, below that, a red trapezium for the pot and, finally, a few circular baubles hanging from the branches. All of these can be described in terms of simple shapes and, therefore, can represented in SVG.</p>
<p>What variables might we want to alter.</p>
<ul>
<li>The number of triangles in the body of the tree.</li>
<li>The colour of the body of the tree</li>
<li>The length of the trunk.</li>
<li>The colour of the bauble..</li>
</ul>
<p>(There are probably more that you can think of, but these will be enough to solve Twinkly's current problem.)</p>
<p>Twinkly set to on this task and quickly managed to write code to draw some shapes that looked like a tree. And then, she bundled the code up into a Moose class, called <a href="https://metacpan.org/module/SVG::ChristmasTree">SVG::ChristmasTree</a>, which she uploaded to CPAN. Like most classes, this class starts by loading Moose and defining some attributes. The most important attribute is the one that contains the SVG object itself.</p>
<pre><code class="code-listing"><span class="word">has</span> <span class="word">svg</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">isa</span> <span class="operator">=></span> <span class="single">'SVG'</span><span class="operator">,</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">lazy_build</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="keyword">sub</span> <span class="word">_build_svg</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$self</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">return</span> <span class="word">SVG</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span><br /> <span class="word">width</span> <span class="operator">=></span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">width</span><span class="operator">,</span><br /> <span class="word">height</span> <span class="operator">=></span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">height</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><span class="structure">}</span></code></pre>
<p>Then there are a number of attributes that define the various characteristics of the tree that we might want to change. Here are some examples:</p>
<pre><code class="code-listing"><span class="word">has</span> <span class="word">layers</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">isa</span> <span class="operator">=></span> <span class="single">'Int'</span><span class="operator">,</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">default</span> <span class="operator">=></span> <span class="number">4</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="word">has</span> <span class="word">trunk_length</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">isa</span> <span class="operator">=></span> <span class="single">'Str'</span><span class="operator">,</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">default</span> <span class="operator">=></span><br /><span class="structure">);</span><br /><br /><span class="word">has</span> <span class="word">leaf_colour</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">isa</span> <span class="operator">=></span> <span class="single">'Str'</span><span class="operator">,</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">default</span> <span class="operator">=></span> <span class="single">'rgb(0,127,0)'</span><span class="operator">,</span><br /><span class="structure">);</span></code></pre>
<p>There's a main driver method that actually draws the tree.</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">as_xml</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$self</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">pot</span><span class="structure">;</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">trunk</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$width</span> <span class="operator">=</span> <span class="number">600</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$tri_bottom</span> <span class="operator">=</span> <span class="number">700</span><span class="structure">;</span><br /> <span class="keyword">for</span> <span class="structure">(</span><span class="number">1</span> <span class="operator">..</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">layers</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$h</span> <span class="operator">=</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">triangle</span><span class="structure">(</span><span class="number">90</span><span class="operator">,</span> <span class="symbol">$width</span><span class="operator">,</span> <span class="symbol">$tri_bottom</span><span class="structure">);</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">bauble</span><span class="structure">(</span><span class="number">500</span> <span class="operator">-</span> <span class="structure">(</span><span class="symbol">$width</span><span class="operator">/</span><span class="number">2</span><span class="structure">)</span><span class="operator">,</span> <span class="symbol">$tri_bottom</span><span class="structure">);</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">bauble</span><span class="structure">(</span><span class="number">500</span> <span class="operator">+</span> <span class="structure">(</span><span class="symbol">$width</span><span class="operator">/</span><span class="number">2</span><span class="structure">)</span><span class="operator">,</span> <span class="symbol">$tri_bottom</span><span class="structure">);</span><br /> <span class="symbol">$width</span> <span class="operator">*=</span> <span class="number">5</span><span class="operator">/</span><span class="number">6</span><span class="structure">;</span><br /> <span class="symbol">$tri_bottom</span> <span class="operator">-=</span> <span class="structure">(</span><span class="symbol">$h</span> <span class="operator">*</span> <span class="float">.5</span><span class="structure">)</span><br /> <span class="structure">}</span><br /><br /> <span class="keyword">return</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">svg</span><span class="operator">-></span><span class="word">xmlify</span><span class="structure">;</span><br /> <span class="structure">}</span></code></pre>
<p>This just calls various lower-level methods which draw the inidividual shapes that make up the tree before, finally, calling <code>xmlify</code> on the SVG attribute which actually produces the XML output that describes the tree.</p>
<p>As most of the parts of the tree are just coloured shapes, there's a method called <code>coloured_shape</code> which does most of that work.</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">coloured_shape</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$self</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$x</span><span class="operator">,</span> <span class="symbol">$y</span><span class="operator">,</span> <span class="symbol">$colour</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$path</span> <span class="operator">=</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">svg</span><span class="operator">-></span><span class="word">get_path</span><span class="structure">(</span><br /> <span class="word">x</span> <span class="operator">=></span> <span class="symbol">$x</span><span class="operator">,</span><br /> <span class="word">y</span> <span class="operator">=></span> <span class="symbol">$y</span><span class="operator">,</span><br /> <span class="word">-type</span> <span class="operator">=></span> <span class="single">'polyline'</span><span class="operator">,</span><br /> <span class="word">-closed</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">svg</span><span class="operator">-></span><span class="word">polyline</span><span class="structure">(</span><br /> <span class="cast">%</span><span class="symbol">$path</span><span class="operator">,</span><br /> <span class="word">style</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="word">fill</span> <span class="operator">=></span> <span class="symbol">$colour</span><span class="operator">,</span><br /> <span class="word">stroke</span> <span class="operator">=></span> <span class="symbol">$colour</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">);</span><br /> <span class="structure">}</span></code></pre>
<p>This takes three arguments: an array of X points, an array of Y points and a colour. It uses the <a href="https://metacpan.org/module/SVG">SVG</a> modules, <code>get_path</code> and <code>polyline</code> methods to turn the X and Y co-ordinates into an SVG polyline element which it then adds to the SVG image.</p>
<p>Finally, one of the methods that uses <code>coloured_shape</code> looks like this. This is the <code>pot</code> method (so notice that, currently, the co-ordinates are all fixed values).</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">pot</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$self</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">coloured_shape</span><span class="structure">(</span><br /> <span class="structure">[</span> <span class="number">400</span><span class="operator">,</span> <span class="number">350</span><span class="operator">,</span> <span class="number">650</span><span class="operator">,</span> <span class="number">600</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">[</span> <span class="number">1000</span><span class="operator">,</span> <span class="number">800</span><span class="operator">,</span> <span class="number">800</span><span class="operator">,</span> <span class="number">1000</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="symbol">$self</span><span class="operator">-></span><span class="word">pot_colour</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><span class="structure">}</span></code></pre>
<p>To create a Christmas tree you just need to create an object and call its <code>as_xml</code> method:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">SVG::ChristmasTree</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$tree</span> <span class="operator">=</span> <span class="word">SVG::ChristmasTree</span><span class="structure">;</span><br /><span class="word">print</span> <span class="symbol">$tree</span><span class="operator">-></span><span class="word">as_xml</span><span class="operator">.</span></code></pre>
<p>Which renders in the browser like so:</p>
<center><img src="tree.svg" width="500" height="436"></center>
<p>If you want to vary the attributes, you do that as you create the object.</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$tree</span> <span class="operator">=</span> <span class="word">SVG::ChristmasTree</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">layers</span> <span class="operator">=></span> <span class="number">6</span><span class="operator">,</span><br /> <span class="word">leaf_colour</span> <span class="operator">=></span> <span class="single">'rgb(0,255,0)'</span><span class="operator">,</span><br /><span class="structure">})</span></code></pre>
<p>(Notice that the colour attributes are defined using SVG's standard RGB notation.)</p>
<p>Twinkly wanted to make her life as easy as possible, so she decided to write a <code>tree</code> program that did all this for her. She was about to write a pile of command-line parsing code in order to handle all of the attributes when she remembered the existance of <code>MooseX::Getopt</code> which did all of that for her. She simple had to add the following line to her class:</p>
<pre><code class="code-listing"><span class="word">with</span> <span class="word">MooseX::Getopt</span><span class="structure">;</span></code></pre>
<p>And, then, in her <code>tree</code> program, she replaced the call to <code>new()</code> with one to <code>new_with_options()</code>. Her code now looks like this.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">SVG::ChristmasTree</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$tree</span> <span class="operator">=</span> <span class="word">SVG::ChristmasTree</span><span class="operator">-></span><span class="word">new_with_options</span><span class="structure">;</span><br /><span class="word">say</span> <span class="symbol">$tree</span><span class="operator">-></span><span class="word">as_xml</span><span class="structure">;</span></code></pre>
<p>And she can call her program like this:</p>
<pre><code> $ ./tree --layers=6 --leaf_colour='rgb(0,255,0)'</code></pre>
<p>The Moose extension handles passing the command line options on to the object constructors. It even knows what the valid options are and displays a helpful message if you try to use an invalid one.</p>
<p>Now, Twinkly just needs to call her <code>tree</code> program with various combinations of command-line options to produce dozens of different trees. And Santa's Christmas cards will be at the printers in time to meet the deadline.</p>
</div>2018-12-12T00:00:00ZDave CrossMuhttp://perladvent.org/2018/2018-12-11.html<div class='pod'><p>There's a day in every elf's life where they learn something that changes the whole way they approach problems from then on.</p>
<p>For Yule Presentgiver that day was today. He was about to learn about <a href="https://metacpan.org/module/Mu">Mu</a>, a way to quickly and compactly write <a href="https://metacpan.org/module/Moo">Moo</a> and <a href="https://metacpan.org/module/Moose">Moose</a> compatible classes with almost no typing.</p>
<p>Yule had written a simple script that read in a CSV file and added up all the present counts in the third column.</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="version">5.12.0</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Text::CSV</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$filename</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$csv</span> <span class="operator">=</span> <span class="word">Text::CSV</span><span class="operator">-></span><span class="word">new</span> <span class="structure">({</span> <span class="word">binary</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">})</span><br /> <span class="operator">or</span> <span class="word">die</span> <span class="double">"Cannot use CSV: "</span><span class="operator">.</span><span class="word">Text::CSV</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /><span class="word">open</span> <span class="word">my</span> <span class="symbol">$fh</span><span class="operator">,</span> <span class="double">"<:encoding(utf8)"</span><span class="operator">,</span> <span class="symbol">$filename</span> <span class="operator">or</span> <span class="word">die</span> <span class="double">"$filename: $!"</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$total</span> <span class="operator">=</span> <span class="number">0</span><span class="structure">;</span><br /><span class="keyword">while</span> <span class="structure">(</span> <span class="keyword">my</span> <span class="symbol">$row</span> <span class="operator">=</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">getline</span><span class="structure">(</span><span class="symbol">$fh</span><span class="structure">)</span> <span class="structure">)</span> <span class="structure">{</span><br /> <span class="symbol">$total</span> <span class="operator">+=</span> <span class="symbol">$row</span><span class="operator">-></span><span class="structure">[</span><span class="number">2</span><span class="structure">];</span><br /><span class="structure">}</span><br /><span class="symbol">$csv</span><span class="operator">-></span><span class="word">eof</span> <span class="operator">or</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /><br /><span class="word">say</span> <span class="symbol">$total</span><span class="structure">;</span></code></pre>
<p>Now he had to take it to the Wise Old Elf for code review.</p>
<p>"That's a nice compact script", The Wise Old Elf commented, "But it's not very reusable. The next ticket has us writing a second script that gives us the total across all the continents, and you can't reuse any of that code you just wrote"</p>
<p>"What would be nice to do would be to put your logic in a reusable module"</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">DeliveryCount</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Moo</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Text::CSV</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">experimental</span> <span class="single">'signatures'</span><span class="structure">;</span><br /><br /><span class="word">has</span> <span class="single">'filename'</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'ro'</span><span class="operator">,</span><br /> <span class="word">required</span> <span class="operator">=></span> <span class="number">1</span><span class="structure">;</span><br /><span class="structure">);</span><br /><br /><span class="word">has</span> <span class="single">'count'</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'lazy'</span><span class="operator">,</span><br /> <span class="word">builder</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="prototype">($self,@)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$csv</span> <span class="operator">=</span> <span class="word">Text::CSV</span><span class="operator">-></span><span class="word">new</span> <span class="structure">({</span> <span class="word">binary</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">})</span><br /> <span class="operator">or</span> <span class="word">die</span> <span class="double">"Cannot use CSV: "</span><span class="operator">.</span><span class="word">Text::CSV</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /> <span class="word">open</span> <span class="word">my</span> <span class="symbol">$fh</span><span class="operator">,</span> <span class="double">"<:encoding(utf8)"</span><span class="operator">,</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">filename</span><br /> <span class="operator">or</span> <span class="word">die</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">filename</span><span class="operator">.</span><span class="double">": $!"</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$total</span> <span class="operator">=</span> <span class="number">0</span><span class="structure">;</span><br /> <span class="keyword">while</span> <span class="structure">(</span> <span class="keyword">my</span> <span class="symbol">$row</span> <span class="operator">=</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">getline</span><span class="structure">(</span><span class="symbol">$fh</span><span class="structure">)</span> <span class="structure">)</span> <span class="structure">{</span><br /> <span class="symbol">$total</span> <span class="operator">+=</span> <span class="symbol">$row</span><span class="operator">-></span><span class="structure">[</span><span class="number">2</span><span class="structure">];</span><br /> <span class="structure">}</span><br /> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">eof</span> <span class="operator">or</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="symbol">$total</span><span class="structure">;</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="number">1</span><span class="structure">;</span></code></pre>
<p>"And then", the Wise Old Elf continued, "you'd just have to write a simple launch script".</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="version">5.12.0</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">DeliveryCount</span><span class="structure">;</span><br /><span class="word">say</span> <span class="word">DeliveryCount</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="word">filename</span> <span class="operator">=></span> <span class="core">shift</span> <span class="structure">)</span><span class="operator">-></span><span class="word">count</span><span class="structure">;</span></code></pre>
<p>"Now you've made that into reusable code the next next ticket is just writing this tiny additional script"</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="version">5.12.0</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">DeliveryCount</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">List::Util</span> <span class="words">qw( sum )</span><span class="structure">;</span><br /><span class="word">say</span> <span class="word">sum</span> <span class="word">map</span> <span class="structure">{</span> <span class="word">DeliveryCount</span><span class="operator">-></span><span class="word">new</span><span class="structure">(</span> <span class="word">filename</span> <span class="operator">=></span> <span class="magic">$_</span> <span class="structure">)</span><span class="operator">-></span><span class="word">count</span> <span class="structure">}</span> <span class="words">qw(<br /> africa.csv antartica.csv asia.csv oceania.csv europe.csv na.csv sa.csv<br />)</span><span class="structure">;</span></code></pre>
<p>"I know, <i>I know</i>.." Yule explained, "but it's soooo much more typing to put the code in a module. All that Moo <b>boilerplate</b>. has-this...lazy that...And we need to move <b>FAST</b>. Christmas is a-comming....".</p>
<p>The Wise Old Elf nodded his head. This was a common theme with Yule. He'd repeatedly shyed away from writing OO code in quick and dirty scripts because he saw it as too much work. Of course, as in this cases, those quick and dirty scripts often turned into something bigger and then Yule'd have to not only write the object orintated code he should have written in the first place but also rewrite all his existing code then too. If only there was a way to get him to write the object orintated code just as quickly in the first place...Oh, yes, now the Wise Old Elf knew what to teach Yule.</p>
<p>"You need to learn about <a href="https://metacpan.org/module/Mu">Mu</a>"</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">DeliveryCount</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Mu</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Text::CSV</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">experimental</span> <span class="single">'signatures'</span><span class="structure">;</span><br /><br /><span class="word">ro</span> <span class="single">'filename'</span><span class="structure">;</span><br /><span class="word">lazy</span> <span class="single">'count'</span> <span class="operator">=></span> <span class="keyword">sub</span> <span class="prototype">($self,@)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$csv</span> <span class="operator">=</span> <span class="word">Text::CSV</span><span class="operator">-></span><span class="word">new</span> <span class="structure">({</span> <span class="word">binary</span> <span class="operator">=></span> <span class="number">1</span> <span class="structure">})</span><br /> <span class="operator">or</span> <span class="word">die</span> <span class="double">"Cannot use CSV: "</span><span class="operator">.</span><span class="word">Text::CSV</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /> <span class="word">open</span> <span class="word">my</span> <span class="symbol">$fh</span><span class="operator">,</span> <span class="double">"<:encoding(utf8)"</span><span class="operator">,</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">filename</span><br /> <span class="operator">or</span> <span class="word">die</span> <span class="symbol">$self</span><span class="operator">-></span><span class="word">filename</span><span class="operator">.</span><span class="double">": $!"</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$total</span> <span class="operator">=</span> <span class="number">0</span><span class="structure">;</span><br /> <span class="keyword">while</span> <span class="structure">(</span> <span class="keyword">my</span> <span class="symbol">$row</span> <span class="operator">=</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">getline</span><span class="structure">(</span><span class="symbol">$fh</span><span class="structure">)</span> <span class="structure">)</span> <span class="structure">{</span><br /> <span class="symbol">$total</span> <span class="operator">+=</span> <span class="symbol">$row</span><span class="operator">-></span><span class="structure">[</span><span class="number">2</span><span class="structure">];</span><br /> <span class="structure">}</span><br /> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">eof</span> <span class="operator">or</span> <span class="symbol">$csv</span><span class="operator">-></span><span class="word">error_diag</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="symbol">$total</span><span class="structure">;</span><br /><span class="structure">};</span><br /><br /><span class="number">1</span><span class="structure">;</span></code></pre>
<p>"Mu is Moo, with less typing", the Wise Old Elf explained. "Whenever you write"</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Mu</span><span class="structure">;</span></code></pre>
<p>"It's just the same as writing"</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Moo</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">MooX::ShortHas</span><span class="structure">;</span></code></pre>
<p>"<a href="https://metacpan.org/module/MooX::ShortHas">MooX::ShortHas</a> exports some extra syntatic sugar that gives you this <code>ro</code> and <code>lazy</code> syntax. Instead of having to write the long <code>has</code> statements you can just use the much shorter syntax."</p>
<p>Yule's face lit up. The <code>ro 'filename'</code> wasn't any more code that writing <code>my $filename</code>. The <code>lazy 'count' => sub ($self,@)</code> wasn't really any longer than writing a simple subroutine declaration. This was Moo without all the overhead!</p>
<p>The Wise Old Elf went on to explain the best thing - because Mu was just a shortcut for Moo, all the classes that are created are compatible with Moo, and therefore are compatible with Moose if and when you use it. And the <code>has</code> syntax is still avaible in Mu scripts - so you can always use it if there's something better written in that style.</p>
<p>From that day on Yule Presentgiver was a different elf. He quickly produced code that was both compact and reusable with a song in his heart and a jolly old grin on his face.</p>
</div>2018-12-11T00:00:00ZMark FowlerChristmas Tree Babbleshttp://perladvent.org/2018/2018-12-10.html<div class='pod'><p>Yesterday's exercise in getting a script runnable on an older version of perl had introduced the elves to App::sigfix, a tool that could transform their source code so that the subroutine signatures they'd been using would be removed and replaced with shim code that could allow their transformed code to run on older versions of perl.</p>
<p>App::sigfix was just the forerunner of several similar tools that they were starting to learn about and take advantage of.</p>
<h3 id="Babel">Babel</h3>
<p>As much as the elves loved Perl, they, like most programmers, actually coded in several languages. One of their most used was JavaScript. JavaScript as a programming language has evolved over the years, but if it still has to run on older webbrowsers the code that is served on the webserver has to be written in the old and outdated dialect of JavaScript.</p>
<p>To help with this in the JavaScript world they have a tool called <a href="https://babeljs.io/">Babel</a>. Babel is able to translate modern idiomatic ES6 style JavaScript code like this:</p>
<pre><code class="code-listing">($ => $(() => <span class="synIdentifier">{</span><br /> <span class="synStatement">const</span> foo = $(<span class="synConstant">'#foo'</span>)<br /> foo.on(<span class="synConstant">'click'</span>, () => console.log(<span class="synConstant">'foo'</span>))<br /><span class="synIdentifier">}</span>))(jQuery)</code></pre>
<p>Into more traditional JavaScript like this:</p>
<pre><code class="code-listing">(<span class="synIdentifier">function</span> ($) <span class="synIdentifier">{</span><br /> <span class="synStatement">return</span> $(<span class="synIdentifier">function</span> () <span class="synIdentifier">{</span><br /> <span class="synIdentifier">var</span> foo = $(<span class="synConstant">'#foo'</span>);<br /> foo.on(<span class="synConstant">'click'</span>, <span class="synIdentifier">function</span> () <span class="synIdentifier">{</span><br /> <span class="synStatement">return</span> console.log(<span class="synConstant">'foo'</span>);<br /> <span class="synIdentifier">}</span>);<br /> <span class="synIdentifier">}</span>);<br /><span class="synIdentifier">}</span>)(jQuery);</code></pre>
<p>Typically this is run with a tool like gulp whenever the JavaScript file is saved (or whatever other flavor of the month is in vogue in the JavaScript world this week), producing a shippable version that can be tested in the browser.</p>
<h3 id="Babble">Babble</h3>
<p>Babble is a Perl framework for essentially doing the same thing - taking Perl code and transpiling it into Perl code that can run on older versions of perl.</p>
<p>It can run in various modes. For example, it can run in a kind of Just in Time mode where the source code is transformed on the client computer (Babble itself along with its non-core dependencies is written in pure Perl code suitable for running on very old versions of perl):</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="comment"># enable the ::DefinedOr filter to change all the //<br /></span><span class="keyword">use</span> <span class="word">Babble::Filter</span> <span class="words">qw(::DefinedOr)</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$name</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">()</span> <span class="operator">//</span> <span class="double">"World"</span><span class="structure">;</span><br /><span class="word">print</span> <span class="double">"Hello $name\n"</span><span class="structure">;</span></code></pre>
<p>The Babble::Filter command transforms the source code as it's executed so this code will run versions of perl that don't support the defined-or operator <code>//</code>. In fact, the code that perl will see when it executes the script will actually look like this:</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="comment"># enable the ::DefinedOr filter to change all the //<br /></span><span class="keyword">use</span> <span class="word">Babble::Filter</span> <span class="words">qw(::DefinedOr)</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$name</span> <span class="operator">=</span> <span class="structure">(</span><span class="word">map</span> <span class="operator">+</span><span class="structure">(</span><span class="core">defined</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">)</span> <span class="operator">?</span> <span class="magic">$_</span> <span class="operator">:</span> <span class="double">"World"</span><span class="structure">)</span><span class="operator">,</span> <span class="core">shift</span> <span class="structure">())[</span><span class="number">0</span><span class="structure">];</span><br /><span class="word">print</span> <span class="double">"Hello $name\n"</span><span class="structure">;</span></code></pre>
<p>Which isn't the most readable code...but it's not designed for you, it's designed for very old versions of perl!</p>
<h3 id="Source-Filters">Source Filters?</h3>
<p>Perl has had source filters for countless years now, modules that can programmatically alter the source code as perl compiles. What makes Babble so different?</p>
<p>Normally source filters work by applying a dumb regular expression to the source code. They search for a matching string and then replace it with something else. They have no idea of the context the string is being used in - for example, if the string is in the middle of a string or heredoc - so are often fooled into doing the wrong thing. Worse, multiple source filters don't play well together - source filter A doesn't like the unfiltered input designed to be processed by source filter B, and source filter B doesn't like the unfiltered input designed to be processed by source filter A.</p>
<p>Under the hood Babble uses <a href="https://metacpan.org/module/PPR">PPR</a>, a set of grammars for the Perl programming language. This solves the context problem - rather than searching for a string to replace in isolation, Babble matches the entire document. Babble plugins are also able to play together so you can have multiple plugins transforming the same source; A Babble plugin works by changing the way the grammar is matched and interpreted, meaning essentially Babble ends up transforming the source for all plugins together rather than forcing one plugin to deal with the input for another.</p>
<h3 id="Shipping-Transformed-Code">Shipping Transformed Code</h3>
<p>Earlier I mentioned that you can run Babble in different modes. What's the other option?</p>
<p>Rather than having Babble transform the code in Just In Time mode, you can transpile the source code on your local machine and then ship that transformed code to the end user. This has the advantage that your end user doesn't have to have to install the Babble infrastructure, and it'll speed up startup time for your script.</p>
<p>Transformation of source code is easy:</p>
<pre><code> perl -MBabble::Filter=::DefinedOr -pe babble hello.pl</code></pre>
<p>Since very early versions of perl 5, Perl 5 has had the ability to use <code>.pmc</code> files. If a <code>.pmc</code> file is located in the same directory as a <code>.pm</code> file then perl will load the <code>.pmc</code> file in preference to the <code>.pm</code> file - meaning it's possible to ship <i>Perl module compiled</i> code that you've created with Babble alongside the original source code.</p>
<pre><code> perl -MBabble::Filter=::DefinedOr -pe babble MyModule.pm > MyModule.pmc</code></pre>
<p>Of course, running the above command for each module you want to ship will get tedious soon, and is an error prone technique (you're likely forget once and ship a <code>.pmc</code> that's not a transformed version of the <code>.pm</code>) What you need is a build system that handles all of this for you.</p>
<p>The most popular module build system for Perl <a href="https://metacpan.org/module/Dist::Zilla">Dist::Zilla</a> is able to support Babble with the <a href="https://metacpan.org/module/Dist::Zilla::Plugin::Babble">Dist::Zilla::Plugin::Babble</a> plugin. Simply add the following to your <code>dist.ini</code></p>
<pre><code> [Babble]
plugin = ::DefinedOr</code></pre>
<p>And now whenever you execute <code>dzil build</code> Babble will do the transformation for you.</p>
<h3 id="Whats-Possible-with-Babble-today">What's Possible with Babble today?</h3>
<p>Babel is very much an experimental tool, and as such should be treated with extreme care. On an experimental basis Babble can do a bunch of things today:</p>
<dl>
<dt>Babble::Plugin::CoreSignatures</dt>
<dd>
<p>Bring support for subroutine signatures to older versions of perl.</p>
</dd>
<dt>Babble::Plugin::DefinedOr</dt>
<dd>
<p>Bring support for defined-or (<code>//</code>) to older versions of perl.</p>
</dd>
<dt>Babble::Plugin::PostfixDeref</dt>
<dd>
<p>Bring support for postfix dereferencing (<code>->*%</code> and <code>->*@</code>) to older versions of perl.</p>
</dd>
<dt>Babble::Plugin::State</dt>
<dd>
<p>Bring support for the <code>state</code> keyword to older versions of perl.</p>
</dd>
<dt>Babel::Plugin::SKT</dt>
<dd>
<p>Bring support for the <code>try</code> and <code>catch</code> syntax from <a href="https://metacpan.org/module/Syntax::Keyword::Try">Syntax::Keyword::Try</a> without the Syntax::Keyword::Try module. Especially useful for older versions of perl that don't support the pluggable syntax system that Syntax::Keyword::Try relies upon and it can't be installed on.</p>
</dd>
<dt>Babble::Plugin::SubstituteAndReturn</dt>
<dd>
<p>Bring support for substitution regular expressions returning the changes rather than altering their input (<code>s///r</code>) to older versions of perl.</p>
</dd>
</dl>
<h3 id="Back-at-the-North-Pole">Back at the North Pole</h3>
<p>With Babble the elves were able to ship even more of their code to run on third party servers - even those that only supported the ancient perl 5.8. Sure, Babble was experimental, but since the elves were able to do the transformation on the systems they controlled and test the code before they shipped it they had every confidence that what they shipped would work come Christmas Eve.</p>
</div>2018-12-10T00:00:00ZMark FowlerThe Fix Is Inhttp://perladvent.org/2018/2018-12-09.html<div class='pod'><p>One of the distinct advantages of writing code at the north pole is that Santa's elves always deployed the latest versions of perl to their production servers. This meant that the elves were able to make use of the latest Perl features to make their day easier.</p>
<p>In fact, the elves had gone one step further and become addicted to the new subroutine signatures feature of Perl - most of the younger elves would refuse to program without it. While still technically an experimental feature, it'd been available since seemingly forever (as of this Christmas it'd been available not only in the latest major release of perl, but also in the old maintenance release before that, and in the now deprecated major release before <i>that</i>, the now deprecated major release before <i>that too</i>, and even the now deprecated major release before <i>that as well</i>...you get the idea.)</p>
<p>Not only had subroutine signatures been around <i>forever</i>, but they made writing the code <i>so</i> much nicer. Rather than writing something complicated like this:</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">fetch_reindeer</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$self</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$name</span> <span class="operator">=</span> <span class="core">shift</span> <span class="operator">or</span> <span class="word">die</span> <span class="double">"Missing argument!"</span><span class="structure">;</span><br /> <span class="word">die</span> <span class="double">"Extra argument"</span> <span class="word">if</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<p>The elves only had to write this:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">experimental</span> <span class="single">'signatures'</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">fetch_reindeer</span> <span class="prototype">($self, $name)</span> <span class="structure">{</span><br /> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<p>Which was why it was such a shock when they suddenly found themselves unable to use them.</p>
<h3 id="Time-Waits-For-No-Man-or-Elf">Time Waits For No Man (or Elf)</h3>
<p>It'd probably been a mistake outsourcing hosting some of the application code to an organization not so full of the Christmas spirit. But Santa's operation was the definition of a seasonal business, and they'd just run out of capacity.</p>
<p>"Our rules of conduct say you're not allowed to use any experimental features at all. And while you're in our house, you follow our rules", Father Time said. Being the embodiment of 2018 by December he'd changed from the young baby he'd been back in January into the grouchy old man that was enforcing the pointless rules today.</p>
<p>There was nothing for it. The elves would have to go through all their code and rip out the subroutine signatures, replacing them with older style code. What a lot of work! And it's not like they were on a hard deadline or anything!</p>
<h3 id="A-fix">A fix!</h3>
<p>One of the virtues of a Perl programmer is laziness. In particular, never doing by hand what you can get a computer to do for you.</p>
<p>The elves considered writing a Perl program to go through all their Perl code and remove all the signatures and insert comparable code. This, of course, would probably be a Hard problem. Traditionally parsing Perl code is one of the hardest things a Perl program can do, with its user friendly (but not necessarily compiler friendly) syntax making it a collection of nasty edge cases. The question is would the elves be prepared to put in the hard work to be sufficiently lazy?</p>
<p>Luckily the Wise Old Elf had been to The Perl Conference in Glasgow this year and had heard about a tool that could help them: <a href="https://metacpan.org/module/App::sigfix">App::sigfix</a>.</p>
<p><code>sigfix</code> is a command line program that you can run against your Perl code. It'll parse the source code, find signatures, and then transform them into code that executes on older versions of perl. For example given our <code>ReindeerFactory</code> code:</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">ReindeerFactory</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">experimental</span> <span class="single">'signatures'</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">fetch_reindeer</span> <span class="prototype">($self, $name)</span> <span class="structure">{</span><br /> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<p>We can feed it into <code>sigfix</code></p>
<pre><code> shell$ sigfix -i -t plain ReindeerFactory.pm</code></pre>
<p>And get out the transformed version:</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">ReindeerFactory</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><br /><br /><span class="keyword">sub</span> <span class="word">fetch_reindeer</span> <span class="structure">{</span> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$self</span><span class="operator">,</span> <span class="symbol">$name</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /> <span class="operator">...</span><br /><span class="structure">}</span></code></pre>
<p><code>sigfix</code> has worked hard to not change the line numbers code in the source code (so when the elves ship the generated version of their code out to Father Time any error messages will line up with the line numbers in the original version in git). The <code>experimental</code> line has been totally stripped (meaning that this code will work now on older versions of perl that don't ship with that module.)</p>
<p>With the right <code>find</code> command the elves could <code>-exec</code> <code>sigfix</code> on all their source code and make it shippable in seconds.</p>
<h3 id="For-Us-Mere-Mortals">For Us Mere Mortals</h3>
<p>While this is useful for the elves in removing the <i>experimental</i> <code>subroutine</code> feature as a matter of policy, the main advantage for most of us comes from being able to produce code that'll run on older versions of Perl that don't support subroutine signatures. It means that we can use subroutine signatures in our development environments and still produce a shippable version of our code that will run on older perls.</p>
<p>If only there was a way to remove other more modern features like <code>say</code>, <code>//</code>, <code>state</code>, etc too...you know, that sounds like something I could write about in another article...</p>
</div>2018-12-09T00:00:00ZMark FowlerDBD::mysql versus DBD::MariaDBhttp://perladvent.org/2018/2018-12-08.html<div class='pod'><p>To connect to their MySQL database from Perl, the elves had always used the <a href="https://metacpan.org/pod/DBD::mysql">DBD::mysql</a> module. It worked with the MariaDB fork, too.</p>
<p>Recently, the new module <a href="https://metacpan.org/pod/DBD::MariaDB">DBD::MariaDB</a> had been released. There'd been some confusion at the North Pole...Which one should they use?</p>
<p>Minty Cakesprinkler had been to The Perl Conference in Glasgow 2018 and seen a a <a href="http://act.perlconference.org/tpc-2018-glasgow/talk/7509">lightning talk</a> (<a href="https://youtu.be/xGishnF1bQI?t=351">video</a>, <a href="http://choroba.matfyz.cz/18-light">slides</a>), and was able to explain to his fellow elves the advantages of the new module.</p>
<h3 id="Why-you-should-switch">Why you should switch</h3>
<ul>
<li><p>DBD::mysql has problems to compile with the recent MySQL and MariaDB clients (versions 8.0+ and 10.2.4+, respectively).</p>
</li>
<li><p>The original module can use insecure connection even if you requested it connect securely, see <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-10789">CVE-2017-10789</a>. This can lead to a man-in-the-middle attack. The new one refuses to connect if secure connection was requested but it can’t be provided.</p>
</li>
<li><p>The new module fixes <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-10788">CVE-2017-10788</a> which allowed remote attacker to cause a denial of service (use-after-free), also causing test failures on some platforms.</p>
</li>
<li><p>There are several more vulnerabilities that the new module fixes (e.g. workarounds for <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3152">CVE-2015-3152</a>, <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-3305">CVE-2017-3305</a>, or <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-2767">CVE-2018-2767</a>).</p>
</li>
<li><p>Unicode is fully supported in the new module and uses the standard Perl Unicode model, i.e. when writing code you always use decoded Unicode strings. Recent <a href="https://metacpan.org/pod/DBD::Pg">DBD::Pg</a> and <a href="https://metacpan.org/pod/DBD::SQLite">DBD::SQLite</a> implement the same behaviour.</p>
</li>
<li><p>The new module has better support for prepared statements and bind parameters.</p>
</li>
</ul>
<p>For a more detailed overview, visit the <a href="https://metacpan.org/changes/release/PALI/DBD-MariaDB-1.00">Changes</a>.</p>
<h3 id="Why-you-shouldnt-switch">Why you shouldn’t switch</h3>
<ul>
<li><p>The way how DBD::mysql handled Unicode was broken. It’s possible that switching to the new module would break your application, returning suddenly double encoded UTF-8 strings. Fixing a database where some values are encoded incorrectly is an uneasy task and the risk of breaking the data completely might prevent you from making such a decision.</p>
</li>
</ul>
<h3 id="History">History</h3>
<p>If you’d like to understand the reasons why DBD::mysql was forked, you can read <a href="https://www.perlmonks.org/index.pl?node_id=1223996">this thread at PerlMonks</a>. Basically, the most critical fixes were originally applied to DBD::mysql, but because of the reason described here under <i>Why you shouldn’t switch</i>, they were reverted. The <a href="https://www.gooddata.com">GoodData company</a> couldn’t wait for months, so they decided to fork the module.</p>
<p>Various combinations of Perl, server and client versions are continuously tested in <a href="https://travis-ci.org/gooddata/DBD-MariaDB">Travis CI</a>. There were 164 different combinations at the end of November.</p>
<h3 id="Notes">Notes</h3>
<p>You can report any problems or submit your pull requests to the <a href="https://github.com/gooddata/DBD-MariaDB">GitHub repository</a>.</p>
<p>Thanks to Pali for fixing the issues and <a href="https://gooddata.com">GoodData</a> for supporting contribution to open source.</p>
</div>2018-12-08T00:00:00ZE. ChorobaConvert CSV to Pivot Tablehttp://perladvent.org/2018/2018-12-07.html<div class='pod'><p>The Wise Old Elf was working on a project to help refine the naughty-nice algorithm. A child's academic performance wasn't the only thing that they should consider - but someone who'd been working hard at school certainly needed that recognized.</p>
<p>As such, his team was working hard on taking raw data - a bunch of CSV files - and turning them into a pivot table he could use to help work out correlations.</p>
<h3 id="Introducing-Text::CSV::Pivot">Introducing Text::CSV::Pivot</h3>
<p>If you ever wanted to convert data in CSV file to Pivot table then Text::CSV::Pivot is your friend. Let us work through the sample CSV file with content as below:</p>
<pre><code> Student,Subject,Result,Year
"Smith, John",Music,7.0,"Year 1"
"Smith, John",Maths,4.0,"Year 1"
"Smith, John",History,9.0,"Year 1"
"Smith, John",Language,7.0,"Year 1"
"Smith, John",Geography,9.0,"Year 1"
"Gabriel, Peter",Music,2.0,"Year 1"
"Gabriel, Peter",Maths,10.0,"Year 1"
"Gabriel, Peter",History,7.0,"Year 1"
"Gabriel, Peter",Language,4.0,"Year 1"
"Gabriel, Peter",Geography,10.0,"Year 1"</code></pre>
<p>With above data in CSV file, this is what we would expect as result:</p>
<pre><code> Student,Year,Geography,History,Language,Maths,Music
"Gabriel, Peter","Year 1",10.0,7.0,4.0,10.0,2.0
"Smith, John","Year 1",9.0,9.0,7.0,4.0,7.0</code></pre>
<p>You can get the desired output with the help of <a href="https://metacpan.org/module/Text::CSV::Pivot">Text::CSV::Pivot</a>. Before we jump to the solution, let's understand the key parameters to the constructor.</p>
<dl>
<dt>input_file</dt>
<dd>
<p>This would have the input CSV file (full path). This is a mandatory parameter.</p>
</dd>
<dt>col_key_idx</dt>
<dd>
<p>The column index in the input CSV file that you want to be the unique in the result CSV file. The index always starts with 0 (zero). This is a mandatory parameter.</p>
</dd>
<dt>col_name_idx</dt>
<dd>
<p>The column index in the input CSV file that you want to become the column in the result CSV file. This is a mandatory parameter.</p>
</dd>
<dt>col_value_idx</dt>
<dd>
<p>The column index in the input CSV file that would provide the value for the corresponding column with <code>col_name_idx</code>. This is a mandatory parameter.</p>
</dd>
<dt>col_skip_idx</dt>
<dd>
<p>The column index in the input CSV file that you do not want to show up in the result CSV file. This is optional. It expects the index as an array ref.</p>
</dd>
</dl>
<p>Now let's try to get the result as mentioned above. Let's call the input CSV as <code>input.csv</code> having data as mentioned above. This is what you would need to do get the desired result as above.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span> <span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Text::CSV::Pivot</span><span class="structure">;</span><br /><br /><span class="word">Text::CSV::Pivot</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span> <span class="word">input_file</span> <span class="operator">=></span> <span class="single">'input.csv'</span><span class="operator">,</span><br /> <span class="word">col_key_idx</span> <span class="operator">=></span> <span class="number">0</span><span class="operator">,</span><br /> <span class="word">col_name_idx</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="word">col_value_idx</span> <span class="operator">=></span> <span class="number">2</span> <span class="structure">})</span><span class="operator">-></span><span class="word">transform</span><span class="structure">;</span></code></pre>
<p>When you execute the above code, you should get the result in <code>input.pivot.csv</code> file.</p>
<h2 id="CLI">CLI</h2>
<p>The distribution comes with a tool <code>csv-pivot</code> that can be used out of the box without writing a single line of code.</p>
<p>To get the same result as above using the tool <code>csv-pivot</code>, you would run something like below:</p>
<pre><code> $ csv-pivot --input-file input.csv --col-key-idx 0 --col-name-idx 1 --col-value-idx 2</code></pre>
<p>If you want to know the details of all the command line options, then try this:</p>
<pre><code> $ csv-pivot --help
Usage: csv-pivot [OPTIONS]...
OPTIONS:
-i, --i, -input-file, --input-file=s input file (required)
-o, --o, -output-file, --output-file=s output file (optional)
-k, --k, -col-key-idx, --col-key-idx=i key column index (required)
-n, --n, -col-name-idx, --col-name-idx=i name column index (required)
-v, --v, -col-value-idx, --col-value-idx=i value column index (required)
-s, --s, -col-skip-idx, --col-skip-idx=s comma separated skip column index (optional)
-h, --help print this message</code></pre>
<p>Let's try one more example, this time we don't want the column "Year" in the result CSV file. We can easily do that with the tool <code>csv-pivot</code> as below:</p>
<pre><code> $ csv-pivot --input-file input.csv --col-key-idx 0 --col-name-idx 1 --col-value-idx 2 --col-skip-idx 3</code></pre>
<p>You should now have result CSV file <code>input.pivot.csv</code> as below:</p>
<pre><code> Student,Geography,History,Language,Maths,Music
"Gabriel, Peter",10.0,7.0,4.0,10.0,2.0
"Smith, John",9.0,9.0,7.0,4.0,7.0</code></pre>
<h2 id="CONTRIBUTIONS">CONTRIBUTIONS</h2>
<p>If you have any suggestions to improve or found bugs in the code then please raise it on <a href="https://github.com/manwar/Text-CSV-Pivot">GitHub</a>.</p>
</div>2018-12-07T00:00:00ZMohammad S AnwarSticky Delighthttp://perladvent.org/2018/2018-12-06.html<div class='pod'><p>Sugar Plum Pudding was frustrated. Things were going so sloooowly, and there's only so much time an elf can spend drinking eggnog waiting for yet another script to start up before he goes a little bit crazy.</p>
<p>It was his job to validate the updates to the naughty and nice lists that the elfs located in the off-grid field offices would send in. It was a messy process, from extracting thumb drives tied to carrier pigeons to scanning and OCRing messages encoded in the personals section of various publications. This process involved wielding a plethora of handy-dandy scripts that Plum had developed to automate the tedious nature of each of the bespoke data transfer mechanisms.</p>
<p>Before he'd gone on vacation these scripts had been as fast as greased lightning. When he got back a slew of commits from the new intern - Jingle Frosted Cupcake - had made using them practically impossible. Where they used to start up in seconds they now took almost a minute before they even printed out the first welcome line. Inevitably Plum would get bored and distracted waiting for the scripts to do their thing and find himself surfing the internet for funny Reindeer videos rather than doing what he should be doing.</p>
<p>It was obvious to Sugar Plum Pudding that that pesky intern was now somehow loading a whole bunch of code that he didn't really need to at the start of each module via one of the normal <code>use</code> statements. One of these had to be causing half the codebase to be loaded in each and every script, making the whole thing run at the speed of molasses.</p>
<p>Of course, Jingle Frosted Cupcake denied the whole thing.</p>
<p>"Look Sugar Plum Pudding, I know what you're talking about, which is why I instrumented the whole thing and I can tell you for sure, we're not loading anything unexpected."</p>
<p>Plum asked Jingle to show him the code he'd used to instrument things. It was a pretty standard <code>@INC</code> hook as described <a href="http://www.perladvent.org/2017/2017-12-14.html">in last years advent calendar</a></p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">onitorLoading</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Time::HiRes</span> <span class="words">qw(time)</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$time</span> <span class="operator">=</span> <span class="word">time</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">%seen</span><span class="structure">;</span><br /><br /><span class="word">unshift</span> <span class="symbol">@INC</span><span class="operator">,</span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$filename</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$modulename</span> <span class="operator">=</span> <span class="symbol">$filename</span><span class="structure">;</span><br /> <span class="symbol">$modulename</span> <span class="operator">=~</span> <span class="substitute">s{/}{::}g</span><span class="structure">;</span><br /> <span class="symbol">$modulename</span> <span class="operator">=~</span> <span class="substitute">s{[.]pm$}{}</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">if</span> <span class="symbol">$seen</span><span class="structure">{</span><span class="symbol">$modulename</span><span class="structure">};</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$t</span> <span class="operator">=</span> <span class="word">time</span> <span class="operator">-</span> <span class="symbol">$time</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$time</span> <span class="operator">=</span> <span class="word">sprintf</span><span class="structure">(</span><span class="single">'[% 3d.%03d]'</span><span class="operator">,</span><span class="word">int</span><span class="structure">(</span><span class="symbol">$t</span><span class="structure">)</span><span class="operator">,</span><span class="structure">(</span><span class="symbol">$t</span><span class="operator">-</span><span class="word">int</span><span class="structure">(</span><span class="symbol">$t</span><span class="structure">))</span><span class="operator">*</span><span class="number">1000</span><span class="structure">);</span><br /><br /> <span class="symbol">$seen</span><span class="structure">{</span> <span class="symbol">$modulename</span> <span class="structure">}</span> <span class="operator">=</span> <span class="number">1</span><span class="structure">;</span><br /><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"$time $modulename\n"</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="core">undef</span><span class="structure">;</span><br /><span class="structure">};</span><br /><br /><span class="number">1</span><span class="structure">;</span></code></pre>
<p>"So you see," Jingle explained, "we push a code ref to the start of <code>@INC</code> and whenever Perl tries to load a module it runs the code ref first and prints out what it wants to load and the current time"</p>
<pre><code> $ perl -MonitorLoading -e 'use DateTime'
[ 0.000] DateTime
[ 0.000] warnings::register
[ 0.000] namespace::autoclean
[ 0.000] B::Hooks::EndOfScope
[ 0.000] Module::Implementation
[ 0.000] Module::Runtime
[ 0.001] Try::Tiny
[ 0.002] Carp
[ 0.002] overloading
[ 0.005] Sub::Util
[ 0.005] List::Util
[ 0.006] constant
[ 0.007] B::Hooks::EndOfScope::XS
[ 0.008] Variable::Magic
[ 0.008] base
[ 0.008] vars
[ 0.010] Sub::Exporter::Progressive
[ 0.012] namespace::clean
[ 0.012] Package::Stash
[ 0.012] Package::Stash::XS
[ 0.013] namespace::clean::_Util
[ 0.015] DateTime::Duration
[ 0.015] DateTime::Helpers
[ 0.015] Scalar::Util
[ 0.016] DateTime::Types
[ 0.016] parent
[ 0.016] Specio::Exporter
[ 0.016] Specio::Helpers
[ 0.016] overload
[ 0.017] Specio::Registry
[ 0.018] Specio
[ 0.018] Specio::Declare
[ 0.018] Specio::Coercion
[ 0.018] Specio::OO
[ 0.018] B
[ 0.020] Eval::Closure
[ 0.021] MRO::Compat
[ 0.021] mro
[ 0.021] DynaLoader
[ 0.021] Config
[ 0.023] Role::Tiny
[ 0.025] Specio::PartialDump
[ 0.025] utf8
[ 0.026] utf8_heavy.pl
[ 0.026] re
[ 0.029] unicore::Heavy.pl
[ 0.037] unicore::lib::Perl::Print.pl
[ 0.038] Specio::TypeChecks
[ 0.038] Storable
[ 0.038] Log::Agent
[ 0.038] Fcntl
[ 0.041] Role::Tiny::With
[ 0.041] Specio::Role::Inlinable
[ 0.043] Specio::Constraint::Simple
[ 0.043] Specio::Constraint::Role::Interface
[ 0.043] Specio::Exception
[ 0.044] Devel::StackTrace
[ 0.044] Devel::StackTrace::Frame
[ 0.045] File::Spec
[ 0.045] File::Spec::Unix
[ 0.045] Cwd
[ 0.054] Specio::DeclaredAt
[ 0.056] Specio::Library::Builtins
[ 0.056] Specio::Constraint::Parameterizable
[ 0.056] Specio::Constraint::Parameterized
[ 0.061] Ref::Util
[ 0.061] Ref::Util::XS
[ 0.064] Specio::Library::Numeric
[ 0.067] Specio::Library::String
[ 0.068] Sub::Identify
[ 0.069] Specio::Constraint::AnyCan
[ 0.070] Specio::Constraint::Role::CanType
[ 0.074] Specio::Constraint::ObjectIsa
[ 0.075] Specio::Constraint::Role::IsaType
[ 0.078] Specio::Constraint::Enum
[ 0.083] Specio::Constraint::Union
[ 0.085] Specio::Constraint::ObjectCan
[ 0.091] Params::ValidationCompiler
[ 0.091] Params::ValidationCompiler::Compiler
[ 0.091] Params::ValidationCompiler::Exceptions
[ 0.092] Exception::Class
[ 0.092] Exception::Class::Base
[ 0.092] Class::Data::Inheritable
[ 0.095] Class::XSAccessor
[ 0.095] Class::XSAccessor::Heavy
[ 0.102] DateTime::Locale
[ 0.102] DateTime::Locale::Data
[ 0.102] File::ShareDir
[ 0.103] Class::Inspector
[ 0.105] List::MoreUtils
[ 0.105] List::MoreUtils::PP
[ 0.108] List::MoreUtils::XS
[ 0.108] Exporter::Tiny
[ 0.119] Params::Util
[ 0.130] DateTime::Locale::FromData
[ 0.131] DateTime::Locale::Util
[ 0.135] DateTime::TimeZone
[ 0.136] DateTime::TimeZone::Catalog
[ 0.139] DateTime::TimeZone::Floating
[ 0.139] Class::Singleton
[ 0.139] DateTime::TimeZone::OffsetOnly
[ 0.140] DateTime::TimeZone::UTC
[ 0.142] DateTime::TimeZone::Local
[ 0.143] DateTime::TimeZone::OlsonDB::Change
[ 0.153] POSIX
[ 0.155] Tie::Hash
[ 0.158] integer
[ 0.165] DateTime::Infinite</code></pre>
<p>"When I run it against our code base, I don't see a whole lot of our code being loaded at all."</p>
<p>Plum thought that this was odd, so he tried it himself. And Jingle was right, almost none of their code <i>was</i> being listed!</p>
<p>Three mince pies later, Jingle figured out why. He'd opened up one of the scripts and spotted the offending lines:</p>
<pre><code class="code-listing"><span class="comment">#!/usr/bin/perl<br /></span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><br /><span class="comment"># load modules from our local directory too<br /></span><span class="keyword">use</span> <span class="word">FindBin</span> <span class="words">qw( $FindBin::Bin )</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">File::Spec::Functions</span> <span class="words">qw( catdir )</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">lib</span> <span class="word">catdir</span><span class="structure">(</span> <span class="symbol">$FindBin::Bin</span><span class="operator">,</span> <span class="single">'lib'</span> <span class="structure">);</span></code></pre>
<p>The last three lines of the code were telling perl to load modules from the <code>lib</code> directory located in the same directory as the script itself. And it did this by pushing a new directory to the front of <code>@INC</code> - in <b>front</b> of the code ref that <code>onitorLoading</code> had put there, meaning that that code hook was never called!</p>
<p>What Sugar Plum Pudding needed to do was somehow make this first entry of <code>@INC</code> <i>sticky</i> so that it always stayed where it was even if something else was pushed to the start of <code>@INC</code>.</p>
<p>Sugar Plum Pudding knew that you can tie arrays in Perl so that whenever someone reads from or writes to them instead of using a dumb old array perl would execute code. And <code>@INC</code> for these purposes can be tied just like any old array.</p>
<p>So fixing this <i>should</i> be as simple as writing a little Perl code. Or, this being Perl, going to the CPAN for some Perl code that does this for us.</p>
<h3 id="Array::Sticky">Array::Sticky</h3>
<p><a href="https://metacpan.org/module/Array::Sticky">Array::Sticky</a> on the CPAN can help us here. It's able to tie an array so that it has a static head part and a mutable end part.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Array::Sticky</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">@reindeer</span><span class="structure">;</span><br /><span class="word">tie</span> <span class="symbol">@reindeer</span><span class="operator">,</span> <span class="single">'Array::Sticky'</span><span class="operator">,</span><br /> <span class="word">head</span> <span class="operator">=></span> <span class="structure">[</span><span class="single">'Rudolph'</span><span class="structure">]</span><span class="operator">,</span><br /> <span class="word">tail</span> <span class="operator">=></span> <span class="structure">[</span><span class="single">'Dasher'</span><span class="operator">,</span><span class="single">'Prancer'</span><span class="operator">,</span><span class="single">'Vixen'</span><span class="structure">];</span><br /><br /><span class="comment"># add more reindeer. We're only really manipulating<br /># the tail part of the list!<br /></span><span class="word">unshift</span> <span class="symbol">@reindeer</span><span class="operator">,</span> <span class="single">'Comet'</span><span class="structure">;</span><br /><br /><span class="word">say</span> <span class="word">join</span> <span class="single">', '</span><span class="operator">,</span> <span class="symbol">@reindeer</span><span class="structure">;</span><br /><span class="comment"># Rudolph, Comet, Dasher, Prancer, Vixen</span></code></pre>
<p>Note that Rudolph stays at the front of the list, and Comet is only <code>unshift</code>-ed to second place.</p>
<p>We can do the same with <code>@INC</code> to keep the first element static:</p>
<pre><code class="code-listing"><span class="word">tie</span> <span class="symbol">@INC</span><span class="operator">,</span> <span class="single">'Array::Sticky'</span><span class="operator">,</span> <span class="word">head</span> <span class="operator">=></span> <span class="structure">[</span><span class="core">shift</span> <span class="symbol">@INC</span><span class="structure">]</span><span class="operator">,</span> <span class="word">body</span> <span class="operator">=></span> <span class="structure">[</span><span class="symbol">@INC</span><span class="structure">];</span></code></pre>
<p>Or, Sugar Plum Pudding can use <a href="https://metacpan.org/module/Array::Sticky::INC">Array::Sticky::INC</a> from the CPAN, that does exactly that!</p>
<pre><code class="code-listing"><span class="keyword">package</span> <span class="word">onitorLoading</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Array::Sticky::INC</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Time::HiRes</span> <span class="words">qw(time)</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$time</span> <span class="operator">=</span> <span class="word">time</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">%seen</span><span class="structure">;</span><br /><br /><span class="word">unshift</span> <span class="symbol">@INC</span><span class="operator">,</span> <span class="keyword">sub</span> <span class="structure">{</span><br /> <span class="core">shift</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$filename</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$modulename</span> <span class="operator">=</span> <span class="symbol">$filename</span><span class="structure">;</span><br /> <span class="symbol">$modulename</span> <span class="operator">=~</span> <span class="substitute">s{/}{::}g</span><span class="structure">;</span><br /> <span class="symbol">$modulename</span> <span class="operator">=~</span> <span class="substitute">s{[.]pm$}{}</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">if</span> <span class="symbol">$seen</span><span class="structure">{</span><span class="symbol">$modulename</span><span class="structure">};</span><br /><br /> <span class="keyword">my</span> <span class="symbol">$t</span> <span class="operator">=</span> <span class="word">time</span> <span class="operator">-</span> <span class="symbol">$time</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$time</span> <span class="operator">=</span> <span class="word">sprintf</span><span class="structure">(</span><span class="single">'[% 3d.%03d]'</span><span class="operator">,</span><span class="word">int</span><span class="structure">(</span><span class="symbol">$t</span><span class="structure">)</span><span class="operator">,</span><span class="structure">(</span><span class="symbol">$t</span><span class="operator">-</span><span class="word">int</span><span class="structure">(</span><span class="symbol">$t</span><span class="structure">))</span><span class="operator">*</span><span class="number">1000</span><span class="structure">);</span><br /><br /> <span class="symbol">$seen</span><span class="structure">{</span> <span class="symbol">$modulename</span> <span class="structure">}</span> <span class="operator">=</span> <span class="number">1</span><span class="structure">;</span><br /><br /> <span class="word">print</span> <span class="word">STDERR</span> <span class="double">"$time $modulename\n"</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="core">undef</span><span class="structure">;</span><br /><span class="structure">};</span><br /><br /><span class="word">Array::Sticky::INC</span><span class="operator">-></span><span class="word">make_sticky</span><span class="structure">;</span><br /><br /><span class="number">1</span><span class="structure">;</span></code></pre>
<h3 id="Its-full-of-dependencies">It's full of dependencies!</h3>
<p>Sugar Plum made the change and soon spotted the guilty commit that Jingle Frosted Cupcake had made that accidentally loaded a module that loaded a module that loaded a module that loaded their entire web app and the entire <a href="https://metacpan.org/module/DBIx::Class">DBIx::Class</a> layer.</p>
</div>2018-12-06T00:00:00ZMark FowlerData::Rmap - Santa's deepdive in an ocean of stockings.http://perladvent.org/2018/2018-12-05.html<div class='pod'><p>On December 24 each year since the beginning of time (ok, since about 280 AD), Saint Nicholas of Myra (aka Santa Claus) has travelled the world, depositing lollipops in the stockings of nice children and for the naughty ones, a lump of coal.</p>
<p>His age aside, did you ever wonder how he manages to do it? The answer is Alabaster "Ali" Snowball - Elf Administrator of the nice-and-naughty list. But what does he actually do?</p>
<h3 id="The-Night-Before-Christmas">The Night Before Christmas</h3>
<p>Just like your parents did while trying to get you to sleep, I'll let you follow this fairy tale with the happy ending. Ali has a list like:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">@neighbourhood</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Donald'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">3</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'naughty'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Theresa'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">5</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'nice'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Angela'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">17</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'nice'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="operator">...</span><br /><span class="structure">);</span></code></pre>
<p>Then he goes through like <a href="https://gist.github.com/andrewsolomon/425dcb8e6168a73f2f09429754686158">this</a></p>
<pre><code class="code-listing"><span class="keyword">foreach</span> <span class="keyword">my</span> <span class="symbol">$person</span> <span class="structure">(</span><span class="symbol">@neighbourhood</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="word">next</span> <span class="word">unless</span> <span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">age</span><span class="structure">}</span> <span class="operator"><</span> <span class="number">16</span><span class="structure">;</span><br /> <span class="keyword">if</span> <span class="structure">(</span><span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">behaviour</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'nice'</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="word">push</span> <span class="cast">@</span><span class="structure">{</span><span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">}}</span><span class="operator">,</span> <span class="single">'lollipop'</span><span class="structure">;</span><br /> <span class="structure">}</span><br /> <span class="keyword">else</span> <span class="structure">{</span><br /> <span class="word">push</span> <span class="cast">@</span><span class="structure">{</span><span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">}}</span><span class="operator">,</span> <span class="single">'coal'</span><span class="structure">;</span><br /> <span class="structure">}</span><br /><span class="structure">}</span></code></pre>
<p>Or, if his code is as terse as he is in conversation it's more likely <a href="https://gist.github.com/andrewsolomon/841bfb5e2d0f8a1edae3ce9c8f450df7">to be</a>:</p>
<pre><code class="code-listing"><span class="word">map</span> <span class="structure">{</span><br /> <span class="word">push</span> <span class="cast">@</span><span class="structure">{</span><span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">}}</span><span class="operator">,</span><br /> <span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">behaviour</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'nice'</span> <span class="operator">?</span> <span class="single">'lollipop'</span> <span class="operator">:</span> <span class="single">'coal'</span><span class="structure">;</span><br /><span class="structure">}</span><br /><span class="word">grep</span> <span class="structure">{</span> <span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">age</span><span class="structure">}</span> <span class="operator"><</span> <span class="number">16</span> <span class="structure">}</span> <span class="symbol">@neighbourhood</span><span class="structure">;</span></code></pre>
<p>From which we get a neighbourhood of</p>
<pre><code class="code-listing"><span class="structure">(</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Donald'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">3</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'naughty'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[</span><span class="single">'coal'</span><span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Theresa'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">5</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'nice'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[</span><span class="single">'lollipop'</span><span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Angela'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">17</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'nice'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="operator">...</span><br /><span class="structure">)</span></code></pre>
<p>and <i>everyone</i> lives happily ever after (especially Donald).</p>
<h3 id="The-Recursive-Falling-Dream">The Recursive Falling Dream</h3>
<p>Alas, like all good children on the list, you're rather naive. You see, even in 280AD Myra was a boomtown without a simple linear list of properties, and dealing with the whole world in modern times it's an even more complex data structure with snippets like:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">$world</span> <span class="operator">=</span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'USA'</span><span class="operator">,</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'Country'</span><span class="operator">,</span><br /> <span class="word">territories</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="operator">...</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Washington, D.C'</span><span class="operator">,</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'District'</span><span class="operator">,</span><br /> <span class="word">streets</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'Avenue'</span><span class="operator">,</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Pennsylvania'</span><span class="operator">,</span><br /> <span class="word">properties</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'House'</span><span class="operator">,</span><br /> <span class="word">colour</span> <span class="operator">=></span> <span class="single">'White'</span><span class="operator">,</span><br /> <span class="word">inhabitants</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Donald'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">3</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'naughty'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="operator">...</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">];</span></code></pre>
<p>Scary, isn't it? Do you have that queasy, roller-coaster-after-Christmas-pudding feeling?</p>
<p>If you need to traverse deeply nested data where you're not sure of the structure <b>don't reinvent the wheel</b> - <a href="https://metacpan.org/module/Data::Rmap">Data::Rmap</a> has done it <a href="https://gist.github.com/andrewsolomon/7c17372f278d4540da1b69c88e4d0cf4">for you</a></p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Data::Rmap</span> <span class="words">qw/rmap_hash/</span><span class="structure">;</span><br /><br /><span class="word">rmap_hash</span> <span class="structure">{</span><br /> <span class="word">push</span><span class="structure">(</span><span class="cast">@</span><span class="structure">{</span><span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">}}</span><span class="operator">,</span> <span class="word">what_they_deserve</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">))</span><br /> <span class="word">if</span> <span class="word">they_deserve_anything_at_all</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">)</span><br /><span class="structure">}</span> <span class="symbol">$world</span><span class="structure">;</span></code></pre>
<p>In that snippet <code>rmap_hash</code> recursively traverses the data structure and applies the code block to every hash reference it finds. Then it's simply a matter of implemeting these two validation methods:</p>
<pre><code class="code-listing"><span class="keyword">sub</span> <span class="word">they_deserve_anything_at_all</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$person</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">foreach</span> <span class="keyword">my</span> <span class="symbol">$key</span> <span class="structure">(</span><span class="words">qw/name age behaviour stocking/</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">return</span> <span class="word">unless</span> <span class="word">exists</span><span class="structure">(</span><span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="symbol">$key</span><span class="structure">});</span><br /> <span class="structure">}</span><br /> <span class="keyword">return</span> <span class="word">unless</span> <span class="word">ref</span><span class="structure">(</span><span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">})</span> <span class="operator">eq</span> <span class="single">'ARRAY'</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="word">unless</span> <span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">age</span><span class="structure">}</span> <span class="operator"><</span> <span class="number">16</span><span class="structure">;</span><br /> <span class="keyword">return</span> <span class="number">1</span><span class="structure">;</span><br /><span class="structure">}</span><br /><br /><span class="keyword">sub</span> <span class="word">what_they_deserve</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="symbol">$person</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span><br /><br /> <span class="keyword">return</span> <span class="symbol">$person</span><span class="operator">-></span><span class="structure">{</span><span class="word">behaviour</span><span class="structure">}</span> <span class="operator">eq</span> <span class="single">'nice'</span> <span class="operator">?</span> <span class="single">'lollipop'</span> <span class="operator">:</span> <span class="single">'coal'</span><span class="structure">;</span><br /><span class="structure">}</span></code></pre>
<h3 id="Avoiding-the-Trap">Avoiding the Trap</h3>
<p>The problem is that the world is more than just a bunch of countries. Apart from countries there are other entities like:</p>
<pre><code class="code-listing"><span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Jungle'</span><span class="operator">,</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'Company'</span><span class="operator">,</span><br /> <span class="word">staff</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Jeff'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">7</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'naughty'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="word">role</span> <span class="operator">=></span> <span class="single">'CEO'</span><span class="operator">,</span><br /> <span class="structure">}</span><br /> <span class="structure">]</span><span class="operator">,</span><br /><span class="structure">}</span></code></pre>
<p>and</p>
<pre><code class="code-listing"><span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Monaco'</span><span class="operator">,</span><br /> <span class="word">type</span> <span class="operator">=></span> <span class="single">'Tax Haven'</span><span class="operator">,</span><br /> <span class="word">evaders</span> <span class="operator">=></span> <span class="structure">[</span><br /> <span class="structure">{</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="single">'Ebenezer "Bono" Scrooge'</span><span class="operator">,</span><br /> <span class="word">age</span> <span class="operator">=></span> <span class="number">12</span><span class="operator">,</span><br /> <span class="word">behaviour</span> <span class="operator">=></span> <span class="single">'nice'</span><span class="operator">,</span><br /> <span class="word">stocking</span> <span class="operator">=></span> <span class="structure">[]</span><span class="operator">,</span><br /> <span class="word">role</span> <span class="operator">=></span> <span class="single">'Philanthropist'</span><span class="operator">,</span><br /> <span class="structure">}</span><br /> <span class="structure">]</span><span class="operator">,</span><br /><span class="structure">}</span></code></pre>
<p>In this case, rather than wasting treats or Ali's time, we ignore these branches with Data::Rmap's <code>cut</code> method.</p>
<pre><code class="code-listing"><span class="word">rmap_hash</span> <span class="structure">{</span><br /> <span class="word">Data::Rmap::cut</span> <span class="word">if</span> <span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">type</span><span class="structure">}</span> <span class="operator">&&</span> <span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">type</span><span class="structure">}</span> <span class="operator">=~</span> <span class="match">m{^(Company|Tax Haven)$}</span><span class="structure">;</span><br /><br /> <span class="word">push</span><span class="structure">(</span><span class="cast">@</span><span class="structure">{</span><span class="magic">$_</span><span class="operator">-></span><span class="structure">{</span><span class="word">stocking</span><span class="structure">}}</span><span class="operator">,</span> <span class="word">what_they_deserve</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">))</span><br /> <span class="word">if</span> <span class="word">they_deserve_anything_at_all</span><span class="structure">(</span><span class="magic">$_</span><span class="structure">)</span><br /><span class="structure">}</span> <span class="symbol">$world</span><span class="structure">;</span></code></pre>
<h3 id="and-a-Happy-New-Year">...and a Happy New Year</h3>
<p>As you can see, Santa's performance has been improved by skipping large portions of the world, but it's still a CPU intensive trip. Don't recurse through your data if you can avoid it.</p>
<p>If you can't avoid recursion though, don't reinvent the wheel. Let Data::Rmap do the traversal so you can just focus on the reasoning.</p>
</div>2018-12-05T00:00:00ZAndrew SolomonChristmas Quotinghttp://perladvent.org/2018/2018-12-04.html<div class='pod'><p>"Today," Buddy Snowbeard explained to his Elf class, "I'm going to <i>change your world</i>."</p>
<p>"Not by teaching you anything earth shattering. Oh no, because becoming an effective programmer isn't about making giant leaps, from awesome framework to awesome framework. It's about learning the little things each day, that make your life easier."</p>
<p>"Today", Buddy intoned with undue seriousness, "I'm going to teach you how to print out a variable!"</p>
<p>The groans from the class were audible. The class knew this. It'd been covered in Perl Programming 101 back in Elf kindergarten. They were sixth graders for Santa's sake, and this was like teaching granny elf to suck candy.</p>
<p>"Now I know what you're all thinking", Buddy continued, "but listen to me for a second."</p>
<p>He tapped the board: "I see a lot of elves out there printing variables out like this:"</p>
<pre><code class="code-listing"><span class="word">say</span> <span class="word">STDERR</span> <span class="single">'DEBUG: The next $childname is '</span><span class="operator">.</span><span class="symbol">$childname</span><span class="structure">;</span></code></pre>
<p>"And you're saying to yourselves what's wrong with that? Plenty! Plenty is wrong with that."</p>
<p>"You in the front, Noelle Toffeecane, what is the problem here?"</p>
<p>"Er," Noelle stammered, "you can't tell where the thing you're printing out starts and ends?"</p>
<p>"Very good Miss Toffeecane. I guess we could change it to something like this:"</p>
<pre><code class="code-listing"><span class="word">say</span> <span class="word">STDERR</span> <span class="single">'DEBUG: The next $childname is "'</span><span class="operator">.</span><span class="symbol">$childname</span><span class="operator">.</span><span class="single">'"'</span><span class="structure">;</span></code></pre>
<p>"Of course this still has problems. What might they be, Cookie Flippyfeast?"</p>
<p>"The childname could have like a double quote in it? You know, it could, like, um mess it up like? So you'll be like, I have no idea where this even starts and ends like?"</p>
<p>"Eloquently put Mr Flippyfeast. And why might <code>$childname</code> have a double quote in it? Because it could <i>like</i> have anything in it <i>like</i>. After all that's why you're printing it out. To debug it. Skittle Dreamwish, what else could be in the string that'd mess up our output?"</p>
<p>"A newline?"</p>
<p>"Good. Another Fruity Flippywarmth?"</p>
<p>"A unicode character"</p>
<p>"Yes! And you'd be trusting your terminal to render the sequence out - you'd have no way of telling exactly what character sequence was actually in the string. What other character might our terminal get confused with, Tiny Plumhat?"</p>
<p>"A control character, sir. You could end up turning the whole terminal red on green. Though that would look festive, sir."</p>
<p>"Indeed, but it wouldn't help with your debugging. In fact, let's create a truly naughty string to print out."</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Term::ANSIColor</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$childname</span> <span class="operator">=</span> <span class="word">color</span><span class="structure">(</span><span class="single">'red on_green'</span><span class="structure">);</span><br /><span class="symbol">$childname</span> <span class="operator">.=</span> <span class="double">"Larry the \N{SNOWMAN}"</span><span class="structure">;</span><br /><span class="symbol">$childname</span> <span class="operator">.=</span> <span class="single">'\n"'</span><span class="structure">;</span></code></pre>
<p>"What do we need to do before we print it out - you in the back - Stripes Dreamfeast isn't it?"</p>
<p>"You need to escape it, sir. I'd use Data::Dumper"</p>
<p>"And how would that look, Mr Dreamfest? Let's write it up on the board."</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Data::Dumper</span><span class="structure">;</span><br /><span class="word">say</span> <span class="word">STDERR</span> <span class="single">'DEBUG: The next $childname is '</span><span class="operator">.</span> <span class="word">Dumper</span><span class="structure">(</span><span class="symbol">$childname</span><span class="structure">)</span></code></pre>
<p>"But even with a nice string that'll print out something like this"</p>
<pre><code> DEBUG: The next $childname is $VAR1 = 'Larry Wall';</code></pre>
<p>"Not exactly what we want. We can do better? Any ideas, Dash Sleepyhat?"</p>
<p>"There are options for Data::Dumper, sir. We can pretty much make it do whatever we want"</p>
<p>"Yes you can. But can <i>you</i> remember them off of the top of your head? I certainly can't. And I don't want to write three lines of options in front of every debug statement I use. What's another module we could use? Anyone?"</p>
<p>The class was silent. Eventually a small elf called Mistle Ivybell raised his hand at the back. "Data::Printer, sir?"</p>
<p>"Data::Printer is a good choice Mr Ivybell, but it wasn't exactly what I was looking for. I'm thinking of something instead that's always installed and ships with perl? Anyone?"</p>
<p>No hands went up.</p>
<p>"Oh surely some of you must have heard of the B module? It allows you to do internal like things with Perl. And one of the things it can do is get a Perl string representation of any string. Let's see how this works:"</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">B</span> <span class="words">qw(perlstring)</span><span class="structure">;</span><br /><span class="word">say</span> <span class="word">STDERR</span> <span class="single">'DEBUG: The next $childname is '</span><span class="operator">.</span> <span class="word">perlstring</span><span class="structure">(</span><span class="symbol">$childname</span><span class="structure">)</span></code></pre>
<p>"And that prints out something - if not nice, since we didn't gave it such a naughty string to start with - at least unambiguously understandable"</p>
<pre><code> DEBUG: The next $childname is "\x{1b}[31;42mLarry the \x{2603}\""</code></pre>
<p>Just then the bell rang.</p>
<p>"That's all for today class! Don't forget your test scripts are due by the end of the week!"</p>
</div>2018-12-04T00:00:00ZMark FowlerREST API and the Babyhttp://perladvent.org/2018/2018-12-03.html<div class='pod'><p>How does Santa manage to deliver all the presents to all the children on one day? The answer is simple: He doesn't. In many countries, there are different creatures responsible for the job (if you're curious, consult the <a href="https://en.wikipedia.org/wiki/List_of_Christmas_and_winter_gift-bringers_by_country">List of Christmas and winter gift bringers</a> for details).</p>
<p>We live in a global world. People often work for foreign companies and sometimes move abroad. In order to satisfy even their children, the gift bringers had to start cooperating. At the beginning, they used <a href="https://metacpan.org/pod/Convert::ASN1">ASN.1</a> to exchange the data, then switched to <a href="https://metacpan.org/pod/XML::Compile::SOAP">XML</a> at the beginning of the current century, to enter the Twenty-tens using <a href="https://metacpan.org/pod/JSON">JSON</a>.</p>
<h3 id="Baby-Jesus-on-Red-Hat-Enterprise-Linux">Baby Jesus on Red Hat Enterprise Linux</h3>
<p>Let's look at Baby Jesus, responsible for the central part of Europe and Latin America. The author of this article is interested in him for two reasons: Baby Jesus brings the Christmas gifts to his children, and the Baby's business runs on Perl.</p>
<p>Until the end of the previous year, Baby Jesus had used Red Hat Enterprise Linux 6. It featured Perl version 5.10.1, and the web service used <a href="https://metacpan.org/pod/JSON::XS">JSON::XS</a> version 2.27 provided by the vendor. When the Christmas 2017 was over, Kris Kringle decided to upgrade the system to RHEL7, which included Perl upgrade to 5.16.3 and JSON::XS to 3.01. Surprisingly, this wasn't smooth at all.</p>
<p>After the upgrade, JSON data were different. Some numbers that were previously not quoted started to appear in double quotes, while others originally quoted lost their quotes sometimes. Perl powered gift bringers didn't have to worry as Perl doesn't care, but there were Java businesses that required the format to stay stable. BJ couldn't but accept their argument that the data should roundtrip: when you POST a JSON, you expect to GET the same JSON back.</p>
<h3 id="System-Perl">System Perl</h3>
<p>The investigation carried out showed two causes of the problem: Perl's internal flags had changed across the versions, as well as the heuristics used by the JSON::XS module to guess whether a number should be quoted or not.</p>
<p>"We told you not to use the system Perl!" shouted other programmers in the office.</p>
<p>"Good advice," replied Baby Jesus, "but it wouldn't have helped us. We'd have probably only discovered the problem earlier when upgrading Perl or the library."</p>
<h3 id="Alternatives">Alternatives</h3>
<p>There are several JSON encoding/decoding modules in the wild. Each of them has its own peculiarities. Baby Jesus compared the behaviour of <a href="https://metacpan.org/pod/JSON::PP">JSON::PP</a>, <a href="https://metacpan.org/pod/Cpanel::JSON::XS">Cpanel::JSON::XS</a>, and <a href="https://metacpan.org/pod/JSON::XS">JSON::XS</a>, the latter under Perls 5.10.1 and 5.16.3.</p>
<h4 id="Bah-Humbug">Bah Humbug!</h4>
<pre><code class="code-listing"><span class="word">encode_json</span><span class="structure">([</span><span class="magic">$$</span><span class="operator">,</span> <span class="double">""</span> <span class="operator">.</span> <span class="magic">$$</span><span class="structure">])</span></code></pre>
<p>JSON::XS in 5.10.1 returns <code>["213","213"]</code>, but <code>[213,"213"]</code> in 5.16.3 (the behaviour of JSON::PP and Cpanel::JSON::XS).</p>
<h4 id="Dag-Gubmit">Dag Gubmit!</h4>
<pre><code class="code-listing"><span class="symbol">$x</span> <span class="operator">=</span> <span class="number">12</span><span class="structure">;</span> <span class="word">utf8::decode</span><span class="structure">(</span><span class="symbol">$x</span><span class="structure">);</span> <span class="word">encode_json</span><span class="structure">([</span><span class="symbol">$x</span><span class="structure">])</span></code></pre>
<p>JSON::XS in 5.10.1 returns <code>[12]</code> in 5.10.1, but <code>["12"]</code> in 5.16.3 (the behaviour of JSON::PP and Cpanel::JSON::XS).</p>
<h4 id="Oh-Chestnuts">Oh Chestnuts!</h4>
<pre><code class="code-listing"><span class="symbol">$x</span> <span class="operator">=</span> <span class="single">'19a'</span><span class="structure">;</span> <span class="symbol">$x</span> <span class="operator">+=</span> <span class="number">0</span><span class="structure">;</span> <span class="word">encode_json</span><span class="structure">([</span><span class="symbol">$x</span><span class="structure">])</span></code></pre>
<p>Here, Cpanel::JSON::XS is the one that differs; it returns <code>[19.0]</code>, while all the others return <code>[19]</code>.</p>
<h4 id="Baubles">Baubles!</h4>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Data::Dumper</span><span class="structure">;</span> <span class="word">Dumper</span><span class="structure">(</span><span class="word">decode_json</span><span class="structure">(</span><span class="single">'[1e4]'</span><span class="structure">))</span></code></pre>
<p>This time, it's JSON::PP who begs to differ, returning <code>[10000]</code> without quotes. All other libraries return the number quoted.</p>
<h4 id="Gift-Wrapped">Gift Wrapped!</h4>
<pre><code class="code-listing"><span class="symbol">$x</span> <span class="operator">=</span> <span class="number">1844674407370955161</span><span class="structure">;</span> <span class="word">encode_json</span><span class="structure">([</span><span class="symbol">$x</span><span class="operator">,</span> <span class="symbol">$x</span> <span class="operator">/</span> <span class="number">10</span><span class="structure">])</span></code></pre>
<p>Another example of JSON::PP being not only slow but also inconsistent with the other libraries. It returns <code>[1844674407370955161,1.84467440737096e+17]</code> contrary to <code>[1.84467440737096e+18,1.84467440737096e+17]</code>.</p>
<h4 id="Ah-Tinsel">Ah Tinsel!</h4>
<p>Tied variables are handled differently in JSON::XS. Consider this snippet:</p>
<pre><code class="code-listing"><span class="comment">#! /usr/bin/perl<br /></span><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><br /><span class="structure">{</span> <span class="keyword">package</span> <span class="word">MyIncrementer</span><span class="structure">;</span><br /> <span class="keyword">use</span> <span class="word">Tie::Scalar</span><span class="structure">;</span><br /> <span class="keyword">use</span> <span class="pragma">parent</span> <span class="word">-norequire</span> <span class="operator">=></span> <span class="single">'Tie::StdScalar'</span><span class="structure">;</span><br /> <span class="keyword">sub</span> <span class="word">TIESCALAR</span> <span class="structure">{</span> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$class</span><span class="operator">,</span> <span class="symbol">$val</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span> <span class="word">bless</span> <span class="cast">\</span><span class="symbol">$val</span><span class="operator">,</span> <span class="symbol">$class</span> <span class="structure">}</span><br /> <span class="keyword">sub</span> <span class="word">FETCH</span> <span class="structure">{</span> <span class="keyword">my</span> <span class="symbol">$s</span> <span class="operator">=</span> <span class="core">shift</span><span class="structure">;</span> <span class="cast">$</span><span class="symbol">$s</span><span class="operator">++</span> <span class="structure">}</span><br /><span class="structure">}</span><br /><br /><span class="keyword">use</span> <span class="word">JSON::XS</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$json</span> <span class="operator">=</span> <span class="single">'JSON::XS'</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">allow_nonref</span><span class="structure">;</span><br /><br /><span class="word">tie</span> <span class="word">my</span> <span class="symbol">$x</span><span class="operator">,</span> <span class="single">'MyIncrementer'</span><span class="operator">,</span> <span class="single">'Xa'</span><span class="structure">;</span><br /><span class="word">print</span> <span class="symbol">$json</span><span class="operator">-></span><span class="word">encode</span><span class="structure">(</span><span class="symbol">$x</span><span class="structure">)</span> <span class="word">for</span> <span class="number">1</span> <span class="operator">..</span> <span class="number">4</span><span class="structure">;</span></code></pre>
<p>JSON::XS returns <code>"Xb""Xd""Xf""Xh"</code> regardless of the Perl version, both other libraries return <code>"Xa""Xb""Xc""Xd"</code>. So it seems JSON::XS calls the <code>FETCH</code> method twice when encoding a value to JSON.</p>
<h4 id="For-My-Sake">For My Sake!</h4>
<p>The behaviour of JSON::XS has changed across Perl versions, too.</p>
<pre><code class="code-listing"><span class="symbol">$j</span> <span class="operator">=</span> <span class="single">'JSON::XS'</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">allow_nonref</span><span class="structure">;</span><br /><span class="symbol">$x</span> <span class="operator">=</span> <span class="number">12</span><span class="structure">;</span><br /><span class="word">print</span> <span class="symbol">$j</span><span class="operator">-></span><span class="word">decode</span><span class="structure">(</span><span class="symbol">$x</span><span class="structure">)</span><span class="operator">,</span> <span class="symbol">$j</span><span class="operator">-></span><span class="word">encode</span><span class="structure">(</span><span class="symbol">$x</span><span class="structure">);</span></code></pre>
<p>In Perl 5.10.1, JSON::XS version 2.27 returns <code>12"12"</code>, while 3.01 returns <code>1212</code>, which is consistent with the other libraries.</p>
<h3 id="Quick-Solution">Quick Solution</h3>
<p>"Let's just quickly fix the data before serialisation," was the initial idea of BJ's team. They wanted to call <code>int</code> on integers, concatenate strings to the empty string, and add floats to zero:</p>
<pre><code class="code-listing"><span class="comment">#! /usr/bin/perl<br /></span><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">JSON::XS</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$integer</span> <span class="operator">=</span> <span class="double">"12"</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$string</span> <span class="operator">=</span> <span class="number">42</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$float</span> <span class="operator">=</span> <span class="double">"122e-1"</span><span class="structure">;</span><br /><br /><span class="word">print</span> <span class="word">encode_json</span><span class="structure">([</span><br /> <span class="word">int</span> <span class="symbol">$integer</span><span class="operator">,</span><br /> <span class="double">""</span> <span class="operator">.</span> <span class="symbol">$string</span><span class="operator">,</span><br /> <span class="number">0</span> <span class="operator">+</span> <span class="symbol">$float</span><br /><span class="structure">]);</span></code></pre>
<p>But when they started changing the code, they realised it wasn't so easy. Nested structures turned out to be hard to track, as they were usually built in steps in different parts of the code. Unintentionally inspecting a value could lead to an encoding error, and there was no easy way how to mark a value as "ready for serialisation". Moreover, the whole endeavour was confusing for non-Perl teams who needed to touch the code occasionally.</p>
<p>Furthermore, a formal API description already existed, so adding the same information to the code felt redundant.</p>
<h3 id="Proper-Solution">Proper Solution</h3>
<p>The final decision was to enforce the types in exactly one place, right before the serialisation. Fortunately, a company located in Christkind's territory of activity also needed to solve the same problem, and they were able to convince Reini Urban, the maintainer of Cpanel::JSON::XS, to include their solution to his distribution.</p>
<p>Both the methods <code>encode</code> and <code>decode</code> now took an optional argument that described the types of the encoded or decoded structure. When encoding, the programmer had to provide the types, when decoding, the second argument had to be writable and would be populated by a structure describing the types.</p>
<p>For example:</p>
<pre><code class="code-listing"><span class="comment">#! /usr/bin/perl<br /></span><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Cpanel::JSON::XS</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Cpanel::JSON::XS::Type</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$type</span> <span class="operator">=</span> <span class="structure">{</span><span class="word">count</span> <span class="operator">=></span> <span class="word">JSON_TYPE_INT</span><span class="operator">,</span><br /> <span class="word">average</span> <span class="operator">=></span> <span class="word">JSON_TYPE_FLOAT</span><span class="operator">,</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="word">JSON_TYPE_STRING</span><span class="operator">,</span><br /> <span class="word">is_enabled</span> <span class="operator">=></span> <span class="word">JSON_TYPE_BOOL</span><span class="operator">,</span><br /> <span class="word">orders</span> <span class="operator">=></span> <span class="word">json_type_arrayof</span><span class="structure">(</span><span class="word">JSON_TYPE_INT</span><span class="structure">)};</span><br /><br /><span class="word">print</span> <span class="single">'Cpanel::JSON::XS'</span><span class="operator">-></span><span class="word">new</span><span class="operator">-></span><span class="word">pretty</span><span class="operator">-></span><span class="word">canonical</span><br /> <span class="operator">-></span><span class="word">encode</span><span class="structure">({</span><span class="word">count</span> <span class="operator">=></span> <span class="single">'12'</span><span class="operator">,</span><br /> <span class="word">average</span> <span class="operator">=></span> <span class="single">'11.2'</span><span class="operator">,</span><br /> <span class="word">name</span> <span class="operator">=></span> <span class="number">100</span> <span class="operator">/</span> <span class="number">3</span><span class="operator">,</span><br /> <span class="word">is_enabled</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="word">orders</span> <span class="operator">=></span> <span class="structure">[</span> <span class="number">1</span> <span class="operator">..</span> <span class="number">10</span> <span class="structure">]</span><br /> <span class="structure">}</span><span class="operator">,</span> <span class="symbol">$type</span><span class="structure">);</span></code></pre>
<p>Which returns</p>
<pre><code> {
"average" : 11.2,
"count" : 12,
"is_enabled" : true,
"name" : "33.3333333333333",
"orders" : [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10
]
}</code></pre>
<p>The module <a href="https://metacpan.org/pod/Cpanel::JSON::XS::Type">Cpanel::JSON::XS::Type</a> exported the constants <code>JSON_TYPE_INT</code>, <code>JSON_TYPE_STRING</code> and similar, as well as functions like <code>json_type_arrayof</code> to declare types of nested structures.</p>
<p>The decoder worked in the same fashion:</p>
<pre><code class="code-listing"><span class="comment">#! /usr/bin/perl<br /></span><span class="keyword">use</span> <span class="pragma">warnings</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">strict</span><span class="structure">;</span><br /><br /><span class="keyword">use</span> <span class="word">Cpanel::JSON::XS</span><span class="structure">;</span> <span class="keyword">use</span> <span class="word">Cpanel::JSON::XS::Type</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$struct</span> <span class="operator">=</span> <span class="single">'Cpanel::JSON::XS'</span><span class="operator">-></span><span class="word">new</span><br /> <span class="operator">-></span><span class="word">decode</span><span class="structure">(</span><span class="single">'[null,1,1.1,"1",[0],true]'</span><span class="operator">,</span> <span class="word">my</span> <span class="symbol">$type</span><span class="structure">);</span></code></pre>
<p>And the variables then contained</p>
<pre><code class="code-listing"><span class="symbol">$struct</span> <span class="operator">=</span> <span class="structure">[</span> <span class="core">undef</span><span class="operator">,</span> <span class="number">1</span><span class="operator">,</span> <span class="single">'1.1'</span><span class="operator">,</span> <span class="single">'1'</span><span class="operator">,</span> <span class="structure">[</span><span class="number">0</span><span class="structure">]</span><span class="operator">,</span> <span class="number">1</span> <span class="structure">];</span><br /><span class="symbol">$type</span> <span class="operator">=</span> <span class="structure">[</span> <span class="number">256</span><span class="operator">,</span> <span class="number">2</span><span class="operator">,</span> <span class="number">3</span><span class="operator">,</span> <span class="number">4</span><span class="operator">,</span> <span class="structure">[</span><span class="number">2</span><span class="structure">]</span><span class="operator">,</span> <span class="number">1</span> <span class="structure">];</span></code></pre>
<p>where the meanings of the constants could be found in the source code of the XS file (they should be exported by an independent module so other JSON libraries could use them, too):</p>
<pre><code class="code-listing"><span class="synComment">/* types */</span><br /><span class="synPreProc">#define JSON_TYPE_SCALAR </span><span class="synConstant">0x0000</span><br /><span class="synPreProc">#define JSON_TYPE_BOOL </span><span class="synConstant">0x0001</span><br /><span class="synPreProc">#define JSON_TYPE_INT </span><span class="synConstant">0x0002</span><br /><span class="synPreProc">#define JSON_TYPE_FLOAT </span><span class="synConstant">0x0003</span><br /><span class="synPreProc">#define JSON_TYPE_STRING </span><span class="synConstant">0x0004</span><br /><span class="synComment">/* flags */</span><br /><span class="synPreProc">#define JSON_TYPE_CAN_BE_NULL </span><span class="synConstant">0x0100</span><br /><span class="synComment">/* null type */</span><br /><span class="synPreProc">#define JSON_TYPE_NULL JSON_TYPE_CAN_BE_NULL</span></code></pre>
<p>When modelling family relations, Ježíšek used trees. The description of a tree structure is tricky, though, as it leads to a cyclic reference in its type specification (there are no cycles in a tree, but a child of a node is again a node), which causes a memory leak:</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Cpanel::JSON::XS</span><span class="structure">;</span> <span class="keyword">use</span> <span class="word">Cpanel::JSON::XS::Type</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$node</span> <span class="operator">=</span> <span class="structure">{</span><span class="word">value</span> <span class="operator">=></span> <span class="word">JSON_TYPE_STRING</span><span class="structure">};</span><br /><span class="symbol">$node</span><span class="operator">-></span><span class="structure">{</span><span class="word">children</span><span class="structure">}</span> <span class="operator">=</span> <span class="word">json_type_arrayof</span><span class="structure">(</span><span class="symbol">$node</span><span class="structure">);</span></code></pre>
<p>The proper way to describe a recursive structure is to use the <code>json_type_weaken</code> function:</p>
<pre><code class="code-listing"><span class="symbol">$node</span><span class="operator">-></span><span class="structure">{</span><span class="word">children</span><span class="structure">}</span> <span class="operator">=</span> <span class="word">json_type_arrayof</span><span class="structure">(</span><br /> <span class="word">json_type_weaken</span><span class="structure">(</span><span class="symbol">$node</span><span class="structure">)</span><br /><span class="structure">);</span></code></pre>
<h3 id="Similar-Stories">Similar Stories</h3>
<p>El Niño Dios wasn't the only one to encounter the problem. See for example <a href="https://www.perlmonks.org/?node_id=1210167">Did the JSON module change?</a> on PerlMonks.</p>
<p>Moreover, the problem isn't particular to JSON. Whenever Perl needs to talk to a system with different type system, you might get fall into the same trap. See for example <a href="https://stackoverflow.com/questions/40800205/why-does-dbi-implicitly-change-integers-to-strings">Why does DBI implicitly change integers to strings</a> on StackOverflow.</p>
<p>In Perl, the internal type of a value shouldn't be important. The only exception to this were the bitwise operators, but the introduction of the <code>bitwise</code> feature in 5.22 fixed it, so you can now always specify the type explicitly by using the appropriate operator. Cpanel::JSON::Type follows the same philosophy.</p>
<h3 id="Thanks">Thanks</h3>
<p>The Baby Jesus would like to thank <a href="https://metacpan.org/author/PALI">Pali</a> for implementing the features, <a href="https://metacpan.org/author/RURBAN">Reini Urban</a> for releasing them, and <a href="https://gooddata.com/">GoodData</a> for supporting contribution to open source.</p>
<h3 id="Notes">Notes</h3>
<p>Based on the <a href="http://act.perlconference.org/tpc-2018-glasgow/talk/7340">talk</a> presented at The Perl Conference in Glasgow 2018 (<a href="https://youtu.be/E70b73KlP6E">video</a>, <a href="http://choroba.matfyz.cz/18-yapc">slides</a>).</p>
</div>2018-12-03T00:00:00ZE. ChorobaPerl in the Browserhttp://perladvent.org/2018/2018-12-02.html<div class='pod'><p>"Busted Presents!", Tinsle Gumdrop complained. "I <i>hate</i> re-writing code."</p>
<p>It was Tinsle's job to port a whole bunch of backend validation logic code from their Perl based internet web app to the JavaScript front end so the validation could run client side too.</p>
<p>"I bet those Valentine's cherubs who code in Node don't have this problem", she thought to herself. "At least they can run the same code on the server and in the browser."</p>
<p>If only there was a way to run <i>Perl</i> code within the browser also...</p>
<h3 id="WebPerl">WebPerl</h3>
<p>The JavaScript engines in modern web browsers are crazy powerful optimized insanity - even on occasion shipping an entire LLVM compiler to just in time compile the JavaScript code down into native instructions in order to get native performance.</p>
<p>In order to help get this level of performance you can use <a href="http://asmjs.org/">asm.js</a> - a specialist subset of JavaScript to represent low level operations that the JavaScript engine can easily compile down to native code. Or, if your modern browser supports it (like recent versions of Chrome, Firefox, Safari and IE do) you can skip the JavaScript representation entirely and use a machine independent binary format to describe operations the JavaScript engine should run with <a href="https://webassembly.org/">WebAssembly</a>.</p>
<p>It's possible to compile C code using <a href="http://emscripten.org">Emscripten</a> to produce JavaScript code or WebAssembly that (when further automatically compiled for JIT performance in the browser) will run at near-native speed. Of course, Tinsel doesn't want to run any C based code in the browser - she wants to run Perl code. What you need to run Perl code is perl - the Perl interpreter - which, as luck would have it, is itself a C based program. So using Enscripten we can compile a version of perl that can be used, with a bunch of shims and bridges, to execute Perl code embedded in our HTML source code.</p>
<p>This all sounds like a lot of work - or would be, if <a href="https://www.zero-g.net/#about">Hauke Dämpfling's</a> <a href="https://webperl.zero-g.net/">WebPerl project</a> hadn't done all the hard work for us already. All we need to do to run Perl code is include a single script tag in our HTML linking to a JavaScript file containing the precompiled perl interpreter and suddenly we're able to write Perl code directly in our webpages.</p>
<h3 id="A-working-example">A working example</h3>
<p>For fun, let's write some code to generate Fibonacci numbers in our browser...in Perl.</p>
<pre><code class="code-listing"><span class="synIdentifier"><</span><span class="synStatement">html</span><span class="synIdentifier">></span><br /> <span class="synIdentifier"><</span><span class="synStatement">head</span><span class="synIdentifier">></span><br /><span class="synPreProc"> </span><span class="synIdentifier"><</span><span class="synStatement">script</span><span class="synIdentifier"> </span><span class="synType">src</span><span class="synIdentifier">=</span><span class="synConstant">"https://webperlcdn.zero-g.net/v0.07-beta/webperl.js"</span><br /><span class="synIdentifier"> integrity=</span><span class="synConstant">"sha256-jL8SB7St5ou4+hb0frK0k6VCQXsWQ1wolDrdU7i4juc="</span><br /><span class="synIdentifier"> </span><span class="synType">crossorigin</span><span class="synIdentifier">=</span><span class="synConstant">"anonymous"</span><span class="synIdentifier">></</span><span class="synStatement">script</span><span class="synIdentifier">></span><br /><span class="synPreProc"> </span><span class="synIdentifier"><</span><span class="synStatement">script</span><span class="synIdentifier"> </span><span class="synType">src</span><span class="synIdentifier">=</span><span class="synConstant">"https://code.jquery.com/jquery-3.3.1.min.js"</span><br /><span class="synIdentifier"> integrity=</span><span class="synConstant">"sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="</span><br /><span class="synIdentifier"> </span><span class="synType">crossorigin</span><span class="synIdentifier">=</span><span class="synConstant">"anonymous"</span><span class="synIdentifier">></</span><span class="synStatement">script</span><span class="synIdentifier">></span><br /><span class="synPreProc"> </span><span class="synIdentifier"></</span><span class="synStatement">head</span><span class="synIdentifier">></span><br /> <span class="synIdentifier"><</span><span class="synStatement">body</span><span class="synIdentifier">></span><br /> fib(<br /> <span class="synIdentifier"><</span><span class="synStatement">input</span><span class="synIdentifier"> </span><span class="synType">id</span><span class="synIdentifier">=</span><span class="synConstant">"in"</span><span class="synIdentifier"> </span><span class="synType">type</span><span class="synIdentifier">=</span><span class="synConstant">"number"</span><span class="synIdentifier"> </span><span class="synType">min</span><span class="synIdentifier">=</span><span class="synConstant">"1"</span><span class="synIdentifier"> </span><span class="synType">max</span><span class="synIdentifier">=</span><span class="synConstant">"49"</span><span class="synIdentifier">></span><br /> ) = <span class="synIdentifier"><</span><span class="synStatement">span</span><span class="synIdentifier"> </span><span class="synType">id</span><span class="synIdentifier">=</span><span class="synConstant">"out"</span><span class="synIdentifier">></</span><span class="synStatement">span</span><span class="synIdentifier">></span><br /><br /> <span class="synIdentifier"><</span><span class="synStatement">script</span><span class="synIdentifier"> </span><span class="synType">type</span><span class="synIdentifier">=</span><span class="synConstant">"text/perl"</span><span class="synIdentifier">></span><br /><span class="synSpecial"> use Memoize qw</span>(<span class="synSpecial"> memoize </span>)<span class="synSpecial">;</span><br /><span class="synSpecial"> sub fib </span><span class="synIdentifier">{</span><br /><span class="synSpecial"> </span><span class="synStatement">return</span><span class="synSpecial"> </span>1<span class="synSpecial"> </span><span class="synStatement">if</span><span class="synSpecial"> $_</span><span class="synIdentifier">[</span>0<span class="synIdentifier">]</span><span class="synSpecial"> <= </span>2<span class="synSpecial">;</span><br /><span class="synSpecial"> </span><span class="synStatement">return</span><span class="synSpecial"> fib</span>(<span class="synSpecial">$_</span><span class="synIdentifier">[</span>0<span class="synIdentifier">]</span><span class="synSpecial"> - </span>1)<span class="synSpecial"> + fib</span>(<span class="synSpecial">$_</span><span class="synIdentifier">[</span>0<span class="synIdentifier">]</span><span class="synSpecial"> - </span>2)<span class="synSpecial">;</span><br /><span class="synSpecial"> </span><span class="synIdentifier">}</span><br /><span class="synSpecial"> memoize</span>(<span class="synConstant">'fib'</span>)<span class="synSpecial">;</span><br /><br /><span class="synSpecial"> my $jq = js</span>(<span class="synConstant">'jQuery'</span>)<span class="synSpecial">;</span><br /><span class="synSpecial"> $jq-></span>(<span class="synConstant">'#in'</span>)<span class="synSpecial">->on</span>(<span class="synConstant">'change'</span><span class="synSpecial">, sub </span><span class="synIdentifier">{</span><br /><span class="synSpecial"> $jq-></span>(<span class="synConstant">'#out'</span>)<span class="synSpecial">->text</span>(<br /><span class="synSpecial"> fib</span>(<span class="synSpecial"> $jq-></span>(<span class="synConstant">'#in'</span>)<span class="synSpecial">->val </span>)<br /><span class="synSpecial"> </span>)<span class="synSpecial">;</span><br /><span class="synSpecial"> </span><span class="synIdentifier">}</span>)<span class="synSpecial">;</span><br /><span class="synSpecial"> </span><span class="synIdentifier"></</span><span class="synStatement">script</span><span class="synIdentifier">></span><br /> <span class="synIdentifier"></</span><span class="synStatement">body</span><span class="synIdentifier">></span><br /><span class="synIdentifier"></</span><span class="synStatement">html</span><span class="synIdentifier">></span></code></pre>
<p>Assuming you're reading this article with a recent web browser then this code is entirely runnable directly <a href="fib.html">in the browser</a>.</p>
<p>There's a lot to unpack in this dense example:</p>
<ul>
<li><p>At the very top of the script we load the webperl code from the webperl CDN. We don't even have to host this on our own server.</p>
</li>
<li><p>Once we've done that we're able to use a special form of the <code><script></code> tag to write Perl code in our webpage by using the <code>type="text/perl"</code> attribute.</p>
</li>
<li><p>We're even able to load core modules from within our embedded Perl code. In the example above we load <a href="https://metacpan.org/module/Memoize">Memoize</a> to cache the output of the <code>fib</code> function so our code runs efficiently.</p>
</li>
<li><p>It's possible to run JavaScript within our Perl code using the <code>js(...)</code> function. In the above example we're just evaluating <code>jQuery</code> to get a reference to the jQuery function we can then use to access jQuery within Perl.</p>
</li>
<li><p>The bridge between Perl and JavaScript is pretty flexible. Here we're executing the jQuery function from within Perl, calling a further method on the result on what it returns, and then we're even passing an anonymous Perl subroutine to jQuery as a callback it should execute whenever the input field changes. Remember, jQuery has no idea that it's being called from and itself is calling Perl code at all!</p>
</li>
</ul>
<h3 id="Alls-Well-That-Ends-Well">Alls Well That Ends Well</h3>
<p>In the end Tinsel was able to copy and paste the more complex logic from the backend Perl code without having to rewrite any of it (and avoiding adding additional logic bugs along the way.)</p>
<p>With the extra time she had saved, she was able to take advantage of another Enscripten project - a compile of <a href="http://www.quakejs.com">Quake 3</a> to run in the browser.</p>
</div>2018-12-02T00:00:00ZMark FowlerSign o' The Timeshttp://perladvent.org/2018/2018-12-01.html<div class='pod'><h2 id="Manic-Monday">Manic Monday</h2>
<p>It is a well known fact that all of Santa's Elves are enthusiastic Perl Developers in their free time, contributing regularly to many of the amazing Perl projects we've come to know and love. However, what you might not know is that they are also part of the Elven Perl Monger group, meeting up on every second Thursday of the month for social occasions, and every third Thursday for technical presentations. It has been said (often) that they should have them the other way around, so they have more time to recover from the social occasions!</p>
<p>It was coming towards the end of the year, and The Wise Old Elf was getting around to updating the Elven Perl Monger website with all the dates for 2019. He always kept getting reminded he was really bad with dates, so this year he wanted to get them all right first time.</p>
<p>As per usual, he started to look around CPAN, and came across a little known module, Calendar::List. It seemed to be able to figure out dates, and have some options to help him filter out days he wasn't interested in. After a bit of reading, he got something useful to begin with.</p>
<p>Calendar::List took two date format fields, and a set of configuration parameters. The date fields allowed The Wise Old Elf to use the EPOCH timestamp to order numerically, and the second format field to present the date as he wished. The configuration parameters, enabled him to specify days he didn't want, the number of entries he wanted to return, and make sure he only got back dates from 2019.</p>
<h2 id="Diamonds-And-Pearls">Diamonds And Pearls</h2>
<p>The Wise Old Elf, pleased with himself, proudly showed off his new code at the very next Elven technical meeting.</p>
<pre><code class="code-listing"><span class="keyword">use</span> <span class="word">Calendar::List</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">%config</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="single">'options'</span> <span class="operator">=></span> <span class="number">52</span><span class="operator">,</span><br /> <span class="single">'exclude'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'monday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'tuesday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'wednesday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'friday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'saturday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'sunday'</span> <span class="operator">=></span> <span class="number">1</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'start'</span> <span class="operator">=></span> <span class="single">'01-01-2019'</span><span class="operator">,</span><br /> <span class="single">'end'</span> <span class="operator">=></span> <span class="single">'01-01-2020'</span><span class="operator">,</span><br /><span class="structure">);</span><br /><br /><span class="keyword">my</span> <span class="symbol">%hash</span> <span class="operator">=</span> <span class="word">calendar_list</span><span class="structure">(</span><span class="single">'EPOCH'</span> <span class="operator">=></span> <span class="single">'DDEXT MONTH YYYY'</span><span class="operator">,</span> <span class="cast">\</span><span class="symbol">%config</span><span class="structure">);</span><br /><span class="keyword">my</span> <span class="symbol">%match</span><span class="structure">;</span> <span class="comment"># counts the dates per month;</span><br /><span class="keyword">my</span> <span class="symbol">@dates</span><span class="structure">;</span> <span class="comment"># dates to print out</span><br /><br /><span class="keyword">for</span> <span class="keyword">my</span> <span class="symbol">$item</span> <span class="structure">(</span><span class="word">sort</span> <span class="word">keys</span> <span class="symbol">%hash</span><span class="structure">)</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$mon</span><span class="structure">)</span> <span class="operator">=</span> <span class="symbol">$hash</span><span class="structure">{</span><span class="symbol">$item</span><span class="structure">}</span> <span class="operator">=~</span> <span class="match">/^\w+\W+(\w+)\W+\w+/</span><span class="structure">;</span><br /> <span class="symbol">$match</span><span class="structure">{</span><span class="symbol">$mon</span><span class="structure">}</span><span class="operator">++</span><span class="structure">;</span><br /><br /> <span class="word">push</span> <span class="symbol">@dates</span><span class="operator">,</span> <span class="symbol">$hash</span><span class="structure">{</span><span class="symbol">$item</span><span class="structure">}</span> <span class="operator">.</span> <span class="single">' (Social)'</span> <span class="word">if</span><span class="structure">(</span><span class="symbol">$match</span><span class="structure">{</span><span class="symbol">$mon</span><span class="structure">}</span> <span class="operator">==</span> <span class="number">2</span><span class="structure">);</span><br /> <span class="word">push</span> <span class="symbol">@dates</span><span class="operator">,</span> <span class="symbol">$hash</span><span class="structure">{</span><span class="symbol">$item</span><span class="structure">}</span> <span class="operator">.</span> <span class="single">' (Technical)'</span> <span class="word">if</span><span class="structure">(</span><span class="symbol">$match</span><span class="structure">{</span><span class="symbol">$mon</span><span class="structure">}</span> <span class="operator">==</span> <span class="number">3</span><span class="structure">);</span><br /><span class="structure">}</span></code></pre>
<p>Some of the more learned elves shook their heads, and muttered "He's done it again" to themselves. "What have I done again?" The Wise Old Elf enquired. "You've forgotten the European Perl Conference again, and next year it's in Riga, so we're definitely going!", came the reply. "Oh!" said The Wise Old Elf.</p>
<p>"It's OK", said Sugarplum Stripyboughs, stepping forward, "we only have to change the configuration parameters, and you'll be sorted". A few moments later, she'd made a few changes:</p>
<pre><code class="code-listing"><span class="keyword">my</span> <span class="symbol">%config</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="single">'options'</span> <span class="operator">=></span> <span class="number">52</span><span class="operator">,</span><br /> <span class="single">'exclude'</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">'monday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'tuesday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'wednesday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'friday'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'weekend'</span> <span class="operator">=></span> <span class="number">1</span><span class="operator">,</span><br /> <span class="single">'holidays'</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'07-08-2019'</span><span class="operator">,</span> <span class="single">'08-08-2019'</span><span class="operator">,</span> <span class="single">'09-08-2019'</span> <span class="structure">]</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="single">'start'</span> <span class="operator">=></span> <span class="single">'01-01-2019'</span><span class="operator">,</span><br /> <span class="single">'end'</span> <span class="operator">=></span> <span class="single">'01-01-2020'</span><span class="operator">,</span><br /><span class="structure">);</span></code></pre>
<p>"Note I also changed 'saturday' and 'sunday' to 'weekend' too, saves you a line of code." said Sugarplum. "Excellent, thank you." The Wise Old Elf said with relief. "Now I can update the website."</p>
<h2 id="Around-The-World-In-A-Day">Around The World In A Day</h2>
<p>The Wise Old Elf, after updating the website, sat back and thought how useful that script had been in helping him. "I only need to update a few lines for next year, and I'll be done. Just hope I remember where I put the script!"</p>
</div>2018-12-01T00:00:00ZBarbie