Category Archives: All things Unix

GOTO Considered [Rarely] Necessary

My answer is below, but, surprisingly enough, only a couple of days later, I found it expedient to use a GOTO in a PHP script, admittedly to fix a logic problem that wasn’t very well thought out.  In a while() loop, a test at the end needed to start a new output section, but the current data set needed to be put in the new section.  The expediency was to use the initialization code to start the new section, which was at the top of the loop, but allowing the loop to cycle would read a new data set, so the “quick-and-dirty” solution was to jump to the top of the loop without reading the new value.  Problem solved, and preserved the integrity of the code block, which was the main problem with GOTO in unstructured programs, back in 1968 when Edsgar Dykstra gave his infamous “GOTO Considered Harmful” proclamation.

The only time I have used GOTO has been to implement a decision construct not supported by the language. Of course, GOTO was necessary in linear programming languages which did not have syntactic constructs as we use in structured programming. All compiled programs have a GOTO in each and every looping or decision construct, but in the resulting output code. When it is necessary to use, it must be a construct defined in a safe manner, rigorously applied. Usually, this would be to stop execution in the middle of a code block by a GOTO to exit the block, something which most structured languages already have, in the form of a ‘break’ statement, or similar keyword. I could imagine a deeply nested decision tree construct where GOTO could be used for clarity, but there would probably be another way to write such a construct to be unambiguous.

One such example in C/C++ I could envision would be to construct a type of switch construct that accepted strings instead of integers as case arguments, where GOTO took the place of the break clause. I had used a similar construct in COBOL successfully, to build a switch-type operation, which doesn’t exist in COBOL, such that, once a test succeeded, the rest of the tests would be skipped. (Disclaimer: I took a job writing COBOL as a last resort, around 1990, in the middle of changing careers from systems engineering to software engineering—not a recommended choice: wrote COBOL by day at work and C by night at grad school).


Afterword:  Here is a snippet of the code block…

  
while ($fil = readdir($subdir)) {
newpage:
    if (is_dir($fil)) { continue;}
    if ( $row > 8) {
.
.
.
    if ($column >= 5 ) {
      $column = 1;
      $row++;
      print ("\n");
      goto newpage;
    }
.
.
.

The Curmudgeon Abides

This month marks four years since I finally stopped renewing consulting contracts, which made me officially retired.  Since then, I have continued to maintain my pro bono client lists, and took some time to learn enough Python coding to put up a custom webcam at home, but have let my professional organization memberships lapse and do less coding than ever.  I have continued to create bad videos of our too-infrequent bicycle rides, but the technical skills are gradually eroding without some stimulus to keep up.

Recently, that stimulus came with signing up for the Quora social media site, in which participants can ask questions about random subjects, which get directed to members who have listed some level of expertise in those particular subjects.  So, I get asked questions about software engineering, Linux, operating systems in general, and other related fields which are fading from memory, forcing me to do a bit of research to verify facts I think I know for sure, but which invariably turn out to be not true [or at least, any more].

As a result, my rambling and sometimes oblique discourses on things about which I know very little, but about which I have strong opinions yet, get thrust upon the world, or at least the segment of the Quora community interested in such things.  Some questions are inane, badly formed, or prompt me to ask myself, “People do that now?”  At any rate, a lot of questions get multiple answers, and the answers are ranked by how many members (which may or may not include the original member who posed the question) “upvote” a particular answer.  Quora tends to encourage continued participation by announcing who has upvoted your answer, but that is tempered by statistics showing how many people have seen your answer.  Which, if you weigh that against the paucity of upvotes, means most users glanced at it and moved on.  At least I haven’t seen any “downvotes,” yet.

The social engineering model is fueled by advertising: if users bother to read your post past the opening paragraph, they are greeted by an advertisement before getting to see the rest of the response.  So, Quora has a vested interest in getting lots of responses to questions, and generating lots of questions to be answered.  A large percentage of the questions I get fed are apparently generated by an AI algorithm rather than a real person.  The majority of questions submitted by real people are ones that come from those interested in how to advance in the field, or aspiring programmers curious about pay scales.  Students wonder about the downside of struggling to become a code monkey: how to advance without a formal education or survive in the industry long enough to pay off student loans.  Some, I assume, are looking for answers to class problems without doing their own research.

There are the usual Linux versus Windows arguments, and some loaded questions, possibly posed by ringers to justify promoting a point of view.  A number of the respondents to questions have impressive résumés, and are much better qualified to answer the questions authoritatively than I or many of the others that offer what are clearly biased opinions not grounded in fact.  Many of the questions appear to come from practitioners and aspirants who are in the global marketplace, not many with down-home American monikers like Joe and Charlie, which leads me to fear that the U.S. heartland just isn’t growing a lot of technologists these days, but have relinquished progress to ambitious immigrants and the growing tech sector in the developing world.

So it goes.  Besides keeping my personal technical knowledge base current, and maybe passing on some historical lore to the new generation of coders and admins, I’m preserving a tenuous connection with the community, albeit virtual rather than face-to-face.  However, after a long career of being either the lone “factory rep” at a customer site or the lone Unix guy at a Windows shop, dependent on USENET or other on-line forums for community, it isn’t much different.  It’s as close as I can get to being part of a “Senior Net,” offering advice and guidance as a community service.  And, I get to learn new things, or at least remember the old ones better.

 

 

Internet Purgatory

I’m writing the draft of this post on a word processor (LibreOffice, naturally), for a good reason. We’ve had the same web hosting provider for 17 years, ever since we moved from Missoula to Hamilton in Montana and lost access to ISDN services. For two years before that, we had hosted our own web sites and email from a couple of servers hung from the floor joists, in the basement.

When we needed to find a new home for our Internet presence, Modwest, a new and growing Missoula company, stepped in. The plans they provided were ideal: running on Linux servers, with SSH (Secure Shell) login access and the ability to park multiple domains on one account (at that time, we had two already: parkins.org and info-engineering-svc.com). Everything worked fine, and we added and deleted domains over the years, for ourselves (realizations-mt.com and judyparkins.com) and, temporarily, for clients (onewomandesigns.com). It just worked, and we also added WordPress engines to our personal/business domains. My programming sorted out which domain got served which landing page, and the links from there went to subdomains to keep the filesets separate.

We finally retired the old quilting web site, realizations-mt.com, when the registration expired in early 2018, but rolled the legacy pages into a subdomain of judyparkins.com to keep her client galleries on-line. Then, this spring, Modwest announced they had sold out to Intertune, another web hosting provider headquartered in San Diego. Billing transferred to the new host, with the same pricing. Fine. But, eventually, they told us, the websites and mailboxes would be transferred to the Intertune servers. The Internet thrives on economies of scale—the bigger the organization, the fewer resources are needed for failover, backup, and support.

So, at an inconvenient time (we planned to be out-of-town for a week), they informed us that our parkins.org account would be migrated, so we dutifully switched the collective domains to the new servers, upon which our blogs disappeared, and the other two domains disappeared entirely, along with mail service. Frantic exchanges by phone and email ensued:

Them: “Oh, we’re only migrating parkins.org at this time.”

Us: “But, they share a file set, database, and mailboxes, and have subdomains. It’s one account. And you didn’t migrate the subdomain content at all.”

Them: “Oh, gee, we’ve never seen anything like this. (ed. Note: almost all web hosting services support this.) Switch the others back to Modwest. We’ll get back to you.”

Us: “Unsatisfactory—they are all one and the same, just different names in DNS.”

Us: “Hello? Is anybody there?”

Us: “Our blogs still don’t work, and our mail is scattered across several mail servers.”

Them: “OK, we’ll do what you said, for judyparkins.com and the subdomains.”

Us: “You didn’t. The subdomains sort of work, but the WordPress installation doesn’t, because the three domains are intertwined.”

Them: “OK, try the judyparkins.com now.”

Us: “The blog works, sort of, for Judy’s, but mine doesn’t, and judyparkins.com isn’t receiving mail.

Them: “Oh, wrong mail server. Try it now.”

Us: “OK, now do the same thing for info-engineering-svc.com”

Us: “Hello? Is anybody there? It looks like your servers are pointed the right way, but the email and blogs still don’t work.”

Them: “Oh, wrong mail server. Try it now.”

Us: “OK, the mail works now, but the Larye blog is totally broken: I can’t see blogs.info-engineering-svc.com at all, and the admin page on blogs.parkins.org is broken yet. I can’t publish anything or respond to comments, nada.”

Us: “Hello? Is anybody there?”

Us: “Hello? … Hello?

Now, we could have decided early in this process to not trust them to migrate this successfully and moved everything to a different hosting service, but that would involve a setup fee and transferring all of our files ourselves—which, for the web, isn’t a big deal, but migrating 20,000 emails sitting in hundreds of folders on an IMAP server is—and working through an unfamiliar web control panel, so we didn’t. We should have, as the same level of service, with more space, is actually cheaper on the web hosting service we had before we moved to Montana in 1999.

But, meanwhile,  we’re busy, so a few days passed, and they still hadn’t replied to my latest comm check.  Rather than risk exposing my latent Tourette Syndrome with an expletive-laden outburst via email, I rechecked their servers to see if the info-engineering-svc.com site was working again.  It was, so I switched the domain pointers in ICANN, and waited.  The blog still didn’t work, and I sent another, unacknowledged message to say so.  But, finally, the info-engineering-svc.com blog started working [no reponse from Intertune, but it’s not magic, so they did something], at least in the display mode.  The administrative pages still did not work.

In between guests and visiting relatives, I decided to troubleshoot the WordPress installation, as something was definitely amiss, now that everything was on the same server.  But, changes to the configuration files seemed to have no effect, and a check of the error logs showed the same errors, which didn’t reflect the changes I had made.  A comparison of the installed file set with my backup I had taken before the migration showed no differences: The light begins to dawn, that my blog installation doesn’t match what Intertune is actually serving.

Sure enough, there is a ~/blogs folder, which I had instructed Intertune to configure as the blogs subdomain, and a ~/www/blogs folder, a different version of the wp-admin file set.  After a quick check to make sure that the dozen or two different files in the wp-admin folder were the only differences in the several thousand files in the blog, I copied the version from my backup into the “live” folder, and violá! the admin dashboard appeared, and here we are, composing the rest of the story on-line.

As it turns out, Intertune did not follow my instructions, but did something different, and did not tell me (not for the first time, either).  Somewhere in the middle of the migration, my blog installation got updated, and Judy’s did not, when the blogs were split between Modwest and Intertune, so that the update was only partial, breaking the installation when Intertune took it upon themselves to migrate files I had already manually installed, but to a different location.

So it goes.  One clue was in the WordPress FAQ, which suggested that a HTTP 500 error might be corrected by re-installing the wp-admin directory, which turned out to be the case. Whether we stay with Intertune or not depends on whether we meet any more difficulty with a tech staff that seems incredibly inept, and how much more work we want to do to move our Internet presence to yet another new hosting service, with new rules, and setup fees.  Our old webmail installation no longer works, being reconfigured by Intertune to use their web mail client instead, but we can live with that, and it’s less work for us, though with also less versatility and customization.  Now, to finish tweaking all the PHP-language scripts in the webs for compatibility with PHP 5.3.29.

Raising Software: A Tale of Unruly Tools

One of my retirement hobbies has been shooting and editing videos, mostly of our bicycle adventures.  When I was young, back in the middle of the 20th century, one of my uncles had a movie camera and projector, and I learned to splice broken film and edit clips together with razor blades and transparent tape.  He moved away and I grew up, and the skill was filed away as just another data point in How The World Works.

Fast forward to the second decade of the 21st century, when I decided to strap a cheap digital camera to the handlebars of the bicycle.  Digital editing requires software, a much less messy and more forgiving (i.e., non-destructive to the original “film”) process.  Since we use Linux exclusively as our computing platform of choice, there were a number of choices for Open Source video editing projects, mostly attempts to clone commercial video editing software for Windows and Apple.

I looked at Cinelerra and Kdenlive, which are fairly complex tools with a steep learning curve, but settle on OpenShot, a simpler tool with a lot of attractive features and a cleaner, no frills user interface.  Openshot 1 was essentially a one-man project, a user interface written in Python with the Tk graphical toolkit, built on the Unix principle of lots of little programs working together, using the multimedia command-line editor ffmpeg and associated libraries, the Inkscape vector graphics editor, and the Blender animation tools and libraries.

Openshot 1.4.3

OpenShot made it possible to load up a project with multiple clips, arrange them on a timeline, trim the ends or snip out shorter clips, and add titles and audio tracks, like voice-over and musical scores, in multiple overlaid tracks, and turn video and audio on and off by track and clip.   For several years, this worked fine.  However, success is not always a good thing, and Openshot suffered from it.  Not in the usual riches-to-rags story of an entrepreneur or rock star who descends into excess and loses his way, but in the attempt to seek wider appeal.

OpenShot was originally a purely Linux product, as mentioned.  To port the project to Windows, it was necessary, for a project with limited manpower resources, to keep a common code base.  Openshot attempts to keep a smoothly flowing user interface through parallel processing, using OpenMP.  The Windows philosophy is based on a single-user, single-task model rather than the multi-user, multi-tasking model of Unix.  When Windows evolved into a multi-tasking system, it used the pre-emptive model, which is relatively inefficient for the pipelined processing flow in the Unix cooperative model.  So, Windows applications tend to be monolithic, with all resources loaded in one huge process.  Parallel processing in Windows monolithic applications is accomplished largely through threads, rather the inter-process communication protocols.  I’ve programmed with threads in Linux, which tends to be tricky at best, and takes a thorough knowledge of parallel processing and memory management to do successfully.

The move to Windows-compatible architecture necessitated rewriting a lot of the Unix-specific standard library code in C++, which introduces the danger of memory-management issues. Openshot began to get buggy, with newer versions crashing often.  The developers claim it is the fault of unstable system libraries, but I’m not buying that explanation.  Since the user interface was also getting a major overhaul, work on version 2 meant that no more bug fixes were forthcoming for the now-crippled version 1.4.3.  Alas, initial releases of Version 2, with the back end still largely 1.4 code base, was also prone to crashing as well as presenting an unfamiliar user experience.

Openshot 2.4.1

So, we stayed with version 1.4.3 for a while longer, with short auto-save intervals.  Finally, with crashes and deadlocks rampant, we just had to try version 2 again.  Yes, the crashes had been largely fixed, but the new version was a monolithic package wrapped in a “launcher,”  (AppImage), apparently a type of container tool including all of the library dependencies, which rapidly ate up all available memory and much of the swap space, becoming so slow as to be indistinguishable from deadlock. Memory leaks come to mind when seeing this type of behavior.  On top of that, some of our favorite tools for controlling video blanking and audio muting by track were missing, to be restored by popular demand in a future revision.   Back to 1.4.3 .

Kdenlive

The other alternative, kdenlive, based on the Konquerer Desktop Environment (kde, not native to Ubuntu, thus necessitating loading the complete KDE library support suite), is yet another learning curve, with many editing feature differences and rendering options.  We did use this for one video, as the internal de-shaking algorithm is a bit more efficient than reprocessing the clips with command-line utilities, and we had a bad experience with a new camera mount that was sensitive to shaking.  Kdenlive also crashes from time to time, lending some credence to the Openshot claim that the system libraries are at fault.

But, I continue on, putting up with slow response, freezes, and crashes, because I’m familiar with the features I like, and it produces acceptable videos.   I may spend some time to learn kdenlive, but, hopefully, Openshot 2 will improve over time.  The other alternative is to try to build a native Ubuntu version from source, which is a daunting task, since most open source software has a very specific support software version dependencies.   Despite the woes with Openshot’s growing pains, it is still faster than writing command-line scripts to use the Linux multi-media-processing utilities ffmpeg or avconv to trim and assemble video clips and sound files.  I use those command-line tools to assemble time-lapse videos from my home-built security camera system, but that is much simpler.

Affordable Linux: A Quest

As most readers know, we here at Chaos Central are a Linux establishment for our primary computing.  We use Apple iPhones and iPads for mobile convenience and casual surfing, but depend on Linux for everything else–with the exception of what has become known at CC as “The Screaming Season,” where we are dependent on Windows to run Turbotax to render unto Caesar.

As much as we’d like to be on a 3 to 5-year cycle for computer upgrade/replacement, retirement, and before that, negotiated fee schedules, have reduced that to “when it breaks, MacGyver it.”  The result has been dismal.  When Judy’s desktop Linux machine died last year, having already received a CPU cooling fan transplant several years ago, salvaged from an old machine before it went to the recyclers, we pressed our old 2007-vintage laptop back into service, restoring her files from backup.  That machine had had a new hard drive and more memory installed during its last incarnation, but the extra memory had faded away, and the CPU not up to the demands of the 2016 version of Ubuntu Linux, so it was running slowly with the reduced-capability desktop.

Since the current crop of machines with pre-installed Linux are aimed at developers rather than standard office use, we couldn’t justify a new laptop from one of the few vendors who supply them.  So, as in the “bad old days,” we went shopping for a cheap Windows laptop on which we could install Linux.  The “cheap” machines are now essentially Microsoft’s answer to the iPad and Android tablets they see as main competition–a light-weight, small-screen device with as small (32GB) flash memory (EMMC) instead of a hard drive, and no optical drive.  Not to worry, I loaded a light-weight version of Linux (Lubuntu) on a 64GB flash drive and it became her new machine, with the added advantage of being portable.  The big disadvantage was the need to plug in the Linux memory stick to run it.  The biggest problem was the short life span of flash drives when used as the primary drive on a Unix-like system.   And, the system was incredibly slow.

Well, in only a few months, the flash drive expired.  No problem, I thought, just build another and restore.  But, as many others have found, the ability to install in the first place was a fluke: these little Windows tablets masquerading as a laptop by having an attached keyboard are finicky: try as I might, I could not get it to boot from the install memory stick again.  So, the little blue Dell machine has become the new Turbotax home, effectively retiring the $80 refurbished Vista->Windows7->Windows10 desktop we acquired a couple years ago and built up (another $40 video card) to run Win10 (badly).

So, Judy was back to the old laptop.  This time, we pressed her old 2010 HP Netbook back into service, even slower and more clunky than the older Compaq (which she also used during this evolution), running a light-weight version of Mint Linux.  The office was beginning to look like the computer labs of the early 2000s, where we pressed piles of recycled obsolete machines into experimental compute clusters to get 21st-century performance out of 20th-century machines, a distributed computing architecture conceived in the mid-20th century but not economically feasible until there was a huge supply of retired machines awaiting landfill space.

A comprehensive check of Linux-capable systems was not promising.  The custom-built laptops (our new criteria is portability to fit our mobile retirement life-style) are expensive and beyond our budget with no current paying clients.  My own laptop, which was high-end (and expensive) in its day (2010) is showing its age, with a too-small disk, weak battery and some screen burn-in.  But, the equivalent new models are in excess of $2000, difficult to justify on a pension without contract income to support it.  Nevertheless, it is only a matter of time before it wheezes its last, so we need to first find an affordable and long-term solution for Judy’s immediate need while keeping in mind the future budget hit for a developer machine.  The other option for affordable Linux-ready or pre-installed laptops are refurbished machines proven to be compatible.  The disadvantage there is the same facing my situation: the batteries are in mid-life, so long-term reliability is an issue,as well as mobile portability.  Older machines are also heavier than desired.

So, we finally settled on a fourth option: installing Linux on a Chromebook.  Chrome is a laptop operating system from Google, a variant of the Android system used on smart phones and tablets.  Since Chrome and Android are basically embedded Linux, it is relatively easy to add Linux to them as a container, using Crouton, a management system similar to the Docker system used to install and manage containers on native Linux machines.  I say relatively, but that assumes some heavy-duty skills in system administration, since Chrome is a locked system.  First, the system must be unlocked into Developer mode, which basically voids the warranty and security protections.  At least, Google does not provide support for a machine in this state.  The hacker community, then, is at the mercy of well-intentioned, but not always well-informed peers for advice.

I followed the available advice in the hacker forums into the corners that most others had painted themselves, finally figuring out an important missing step.  To use the Crouton system, the wily hacker must first use the official Google developer shell to set the root and administrator passwords.  Then, Crouton can access the administrative account to install the Linux container.  The less-helpful solutions offered essentially required reinstalling Chrome and starting over, which was not necessary at all, but the default obvious solution to developers and admins raised in the Age of Windows, where frequent rebooting and periodic reinstalling is considered a normal operational necessity.  Unix and Linux admins only reboot after installing a new kernel, which may be months or years, depending on the stability of the system and whether the management decides to incorporate the latest patches when they are released (many don’t, for fear it will break fragile applications, despite the security risk of not upgrading).

So, now Judy has a brand-new system that, hopefully, will be stable for a reasonable time.  The advantage of this system is that it is still Chrome, so she can use the user-friendly desktop apps available to Chrome, similar to the ones on Apple iOS, and switch to the Linux desktop with a simple key-combination to run the Linux applications we depend on.  A similar key-press combination switches back to Chrome.  And, the Chromebook, though large enough to have a full HD display, is light enough to be highly portable, with a very long battery life.  The minor concession to our ill-advised attempt to co-exist with Windows is that we now have a portable tax preparation machine.  During the rest of the year, we may turn it on once in a while to let the updates install and top off the battery, so that it won’t take three days to update next tax season.

Chromebook, running Chrome with Crouton in a browser window.

 

Chromebook, running Ubuntu Linux with LXDE, with a simple browser, and essential apps, including Fiberworks Bronze weaving software running under WINE. (a Windows emulator running in a Linux container under Chrome–boggles the mind, no?)