Among my 50by50 goals, I have a series of inter-related ones called “get my sh** together”. Since those don’t look very nice as page titles, I’ve tweaked it a bit to make it a little more family-friendly (my son, Jacob, age 8, does read my blog, so I should clean up my act). And one thing that has annoyed me on the digital front for a very long time is my online photo gallery.
I won’t bore you with the long version of its sordid history, but the short rant is that I have a website (polywogg.ca) that I pay to host. And so I have my own domain, my own file area, relatively unlimited file storage related to the website (i.e. I can’t turn it into a cloud server, but for the purposes of a website, it’s open season). So I should be able to have a gallery for my personal pictures and video clips. But when it comes to video, almost all the basic hosters have the same limitation.
No hosting video.
If you want to have video on the site, you generally have to upload it to Vimeo or DailyMotion or Youtube (with the ownership and advertisement and privacy issues, oh my, that go along with these sites), and then link to it from your site by embedding the link in your post. It works really well, don’t get me wrong, but dealing with those video sites is a pain in the patootie. A few years ago, I was putting the videos in a password protected area of Vimeo or Dailymotion (I don’t even remember now which one it was), and linking them to a web gallery on my server, running Piwigo as the photo gallery with video embeds. It was working okay, I uploaded about 3000 photos, and then I ran into a problem with the hoster. They claimed it was Piwigo, and after a bunch of testing, frustration, and failure, I eventually killed the Piwigo site and moved everything — photos and video — to SmugMug. Like Flickr and others, it allows you to upload personal photos and videos and then share them. There are basic accounts that are free, but I would have quickly overwhelmed those limits, so I bit the bullet — and paid $80 / year to host everything at SmugMug.
Overall, it’s been great. It took me a while to get up and running, but eventually, I had it all working, so all good. Except it has still been costing me $80 a year when I’m *already paying to host a website elsewhere*. Grrr…oh, and about ten months ago, my workplace updated their firewall blacklist and SmugMug was on it (to prevent people streaming video to the office and sucking up bandwidth) — so when I do blog posts and paste pics from the site, the pics don’t show up in my articles when viewed from work or some other government sites. Which means some of my posts about HR that have pics in them don’t show properly. It has been on my list to fix, but a pain in the patootie to find an alternative, as the most likely alternative is finding somewhere where I could host everything. A new hoster, perhaps. There’s a small chicken-and-egg loop in there, move the site or move the pics, and I’ve not bothered to fix it.
As I said, most basic hosters don’t allow video. They are afraid because video can drastically suck their bandwidth, which they can’t afford to do on basic sites at basic rates. And they don’t want to charge someone $5 / month and have them start streaming movies and trailers for the masses. Except that’s not what I’m doing. I have a few personal videos per month that when posted might get watched by up to five or six people, and then it will sit dormant most of the time. Low bandwidth, nothing major. I know that, the hosters don’t, so they have general policies that say “no video” and they block it internally on the site for all their hosting accounts.
So imagine my surprise recently when I tried a video on my WordPress site and it not only uploaded, stored and loaded on a page, it actually played. WTF? It isn’t blocked? But I know it violates the general terms of service, so I contacted them and asked the question. Maybe I pay them a bit more each month or year so I could host a bit of low bandwidth video on my site? I’m paying SmugMug $80 a year; anything less than that, or potentially even a little more with it fully integrated with my website, and I’d be in digital heaven.
Their response? No extra charge, as long as it is low bandwidth, go ahead. Hallelujah and pass the upload app!
So I confess that once upon a time, I paid for some of my living expenses through the fact that I knew how to work a computer for things like graphics and formatting. No, seriously, back in the early 90s, it made me stand out from other people, the fact that I knew how to work Lotus 1-2-3, Word and WordPerfect, dBase IV and, drumroll please, Harvard Graphics. At the time, I think it was version 3.0, and while people could do very basic graphs in Lotus, the real trick was to choose a variety of simple graphs in Harvard Graphics, add your data, and voila, you were good to go. Back in ’93, for my first job at DFAIT, we did a presentation for Cabinet where all the graphs were done in HG, in colour, and there were about 30 of them. Blew the Cabinet people away, colour slides. In government. Bear in mind that most departments had barely switched over from DOS to Windows, and we gave them a 60 page deck on the future of the Asia Pacific region. Bound.
Now, of course, most people do their graphs in Excel or Powerpoint. And I wanted to do one this week. I have 29 or so “goals” that I am tracking against “Level 1” progress, and I wanted some sort of cheap visualization of how I am doing towards reaching level 1. I started in Powerpoint, which really just replicates the software functionality of Excel, and while it was decent enough, I’m not a graph tweaking expert anymore. I used to be able to get HG get up and dance and look amazing…compared to the Excel graph I did, I feel like I regressed in technology.
So, I was curious. Are there any other data visualization tools out there that people use easily and, more importantly, are free? My friend Aliza mentioned Google Charts and I confess that I had heard it mentioned in passing, but since I had little use for it, I had never looked at it.
I should step back for a second though. I *did* experiment with a WordPress plugin or two a few months back, and well, I was never too satisfied with the result. I would love a simple visualization tool that keeps my data in WordPress, doesn’t add a lot of overhead, and has lots of tweakable options. The experience reminded me a lot of that saying about things being cheap, fast, or good, and you can only pick two. I never quite found what I wanted.
I moved on to a variety of other options:
Zing Chart — highly complex, and the options were not as intuitively laid out as Googel Charts;
Tableau — this is only available for download, and I haven’t tried it yet;
PowerView — a Microsoft option, but not available in the version of Excel I was running;
D3.js — A live data tool, I couldn’t even claim to understand how it worked or what I was supposed to do;
Fusion charts — looks pretty solid, but linking to it from WordPress wasn’t obvious, and thus a dead end;
Visual.IS — in addition to having to create an account, it had strong links to paid versions;
High charts — the type of chart I wanted wasn’t there;
Chartist — good mentions of it elsewhere, but the website is just on GitHub, and not as friendly to access;
Piktochart — although it requires an account to be created, the real challenge was that it is also aimed at infographics more so than simple charts, and for some reason, it starts with asking you about infographics vs. presentation vs. printable, i.e. you’re choosing your design and function before you choose anything about what you want to create and put into said presentation, etc.;
Ubiq — data visualization running with MySQL, and I was hoping to avoid a database element;
Vizydrop — pulls data from a datafile (CSV JSON or Excel), which I was also hoping to avoid, but may reconsider;
Plot.ly — no clue what it did as it just kept crashing;
Datamatic — uses Google Drive, which doesn’t sound problematic, except that GD is blocked by our firewall at work, so visualizations wouldn’t work if I wanted to put them on my screen as a reminder; and,
Hohli — I really liked this one, but it seems to be geared towards replacing the interface (or lack thereof) for Google Charts, yet with 1 time creation (i.e. couldn’t seem to save or edit / re-use later).
Which put me back in the world of Excel. I did a graph, and it looks fine. I have room for future expansion of data, which could be useful (kind of stacking option for Level 2). Not exactly quite what I was hoping for in terms of quality, but maybe that is just me needing to spend more time with Excel to make it pretty. I would LOVE to be able to recreate the DIFF chart look and feel of Google Charts. But it met my need for the day (to update on my progress on goals in Goals – Weekly roundup (#2017-01)).
The other night, I was browsing my posts list in my admin panel on the website and noticed something odd — my scheduled post for yesterday morning didn’t actually post. And right next to it in the internal WordPress admin panel was a nice little red warning called “missed schedule”. I had seen this before and thought it was just a glitch, no biggie — just click publish now and it goes live. No problem with the post, it just didn’t go live when it should have. I don’t often schedule future posts when I’m writing detailed posts, although I might change the time stamp to 8:00 or 9:00 in the morning even if I’m writing it at 1:00 in the afternoon, like now. Just a small quirk I do to manage an indexing protocol that checks the time stamp, no real reason.
However, from time to time, I will write a bunch of posts all at once, and I want them spread out over several days. So I “schedule” them for future publication, usually 9:00 in the morning. So, the question was, “Why didn’t they post?”.
Running around the WordPress community
As with most glitches with WordPress, one is often not the first person to encounter it. Simply googling “WordPress Missed Schedule” brought up a whole series of people who had the same problem, posed the question online and had people respond. In most cases, their problem was slightly different in config once I scratched the surface, or the solutions were a bit different than what would seem to fit my situation. However, they did send me looking into several plugins.
I tried a plugin called WP CRONTROL that wasn’t a fix so much as a way to glean some additional diagnostic information. Namely that the small “glitch” was bigger than I thought — my CRON file was not running, and there were problems going back to August. Hundreds of behind-the-scenes happen on your website without your having to control them directly, little bots that can run from time to time, and your website software or the server software has ways to schedule them. Basically the same as your CALENDAR on your computer keeping track of reminders, etc., or even your regular WINDOWS update that does certain things at regular intervals. In this case, WordPress has a file that keeps track of all things it has to do (its own internal to-do list) called WP-CRON. I started trying to clean out the long list of expired entries but it would take forever.
So I added another cron manager called ADVANCED DATABASE CLEANER. Sure, I could have logged into the PHP admin area of my website and manually cleaned things in my database, but why would I do that when this plugin would do it for me?
Down the rabbit hole
I went down a rabbit hole after that. Lots of little tips and tricks everywhere, but few that were specific enough to help me, nor even in a good logical fashion. More like eight hundred people playing hunches and trying something on their server, some with luck, some with none. I jumped the queue while I was working through some of the options and started chatting with the Server Customer Support Rep, but honestly, their Tier 1 people are more about sales than they are tech support. They do okay, but this is way beyond them.
What I often find helpful though is that explaining the problem to them seems to crystallize it better in my own mind, and often triggers thoughts to try x or y that hadn’t occurred to me earlier or that I dismissed too soon. And it gave me a structure for understanding how to solve the problem.
A. GET YOUR DIAGNOSTICS
As I mentioned above, I used WP CRONTROL to see the list of cron activities and ADVANCED DB CLEANER to fix it a bit. WP CRONTROL also told me there was a 403 error (forbidden) on the WP-CRON file, but it wasn’t very convincing since some of the other errors might be causing it.
After going around the pole a few times, I found a great little tool called WP CRON STATUS CHECKER. It only has a few hundred downloads/installs, and it is a great little tool. The plugin doesn’t have a lot of info with it, but it adds a widget in your dashboard that tells you the status of your CRON files. It DOES nothing to fix problems but it does a quick test to see if WP-CRON can even run, and if not, tells you the status (403, etc.). I wish I had found this first. It told me for sure part of the problem was that WP-CRON was not running. And once fixed, you can deactivate it and leave it off. If you leave it activated, it will run on its own every 24 hours to check, but my site isn’t busy enough that I need that level of monitoring.
B. CHECK .HTACCESS
HTML sites have a small file in the directories called “HT access” that controls how visitors can access the individual files in the directory and subdirectory and is the second line of defense against people doing silly things like writing files and overwriting certain files on your site. On sites like WordPress, it’s the equivalent of a master security file that controls a lot of the subsequent access rights.
Some sites said to see if .HTACCESS was corrupted, but I was pretty sure mine wasn’t (I have other plugins that check the status of HTACCESS regularly, they would throw flags if it was corrupted).
What the sites FAILED to mention was that it isn’t just simple CORRUPTION that could be screwing up WP-CRON, it could be the actual proper setup is messing with it. I should have thought of that immediately when I saw the earlier 403 FORBIDDEN message in WP CRONTROL, but when I saw it again in the WP CRON STATUS CHECKER, I knew I had found my problem. Unfortunately, I did this step sixth or seventh instead of second!
The simplest way to see if .HTACCESS is your problem is to temporarily rename it something like .HTACCESS_TEMP or .HTACCESSXX. Then check the status again — if it was .HTACCESS causing the problem, the status will go green when you check after renaming it. If it is still red, move on to other options.
C. CHECK SECURITY PLUGINS
I run WordFence and iThemes, and of all the types of plugins, security ones are the most likely to cause access issues like above and conflicts with other plugins or activities. So, I temporarily deactivated them and then checked my other settings. Nope, still broken, so that was out.
D. CHECK OTHER PLUGINS
Anyone who has had a problem with WordPress, or really any CMS ever, they know that the simplest way to look for a conflict is to just deactivate all your plugins. Considering some people are running TONS of plugins, this isn’t often the favorite solution, but it’s better than turning them off and on one at a time. I deactivated them all, checked status, still broken.
E. CHECK FILE PERMISSIONS FOR WP-CRON
Of course, lots of people would say to check this one first — if WP-CRON is the problem, the “obvious” first step could be to check the file permissions on that specific file to see if the settings are wrong. Good logic, but since WP sets them in batches, the reality is that other files would likely have the wrong settings too and would have thrown red flags much earlier. Remember that this is a low-level file that only flags errors when it DOESN’T run.
For me, I temporarily changed the setting to 777, full read/write/execute and checked status again. Still broken.
F. CHECK FOLDER PERMISSIONS FOR WP ROOT DIRECTORY
I can’t say this would have occurred to me on my own. If WordPress is running, the folder must be set pretty close to right, or the red flags would be borderline catastrophic for failure. Never the less, it was still worth a try — again, I changed to full 777 rights, no change, still broken.
Why did I do all those things?
If all those had failed, I would have eliminated the three most likely internal problems:
corruption of access files;
access / permissions to the file or folder; and/or
conflict from security or other plugins.
That leaves you three other options that I’ve seen:
Try a plugin called WP MISSED SCHEDULE. It isn’t on the main WP install directory, you have to do it manually from GITHUB. I’m personally leery of installing plugins that don’t reside in the WP repository, and on top of that, the installation instructions did not seem awesome for clarity. Lots of people have tried it and had it fail; others did it and said it worked; others said, “huh?”. I also feel like if the problem was any of the first five, this plugin would not fix any of those issues.
Check with your host to see if there is a server config issue. Lots of people have said very simply, “Oh, it’s a config issue, contact your server”, but that is a bit too reactionary. If it is any of the above issues, the server people won’t know how to fix the problem either and unless you’re paying a lot for access to their tech support personnel, they’ll say, “Sorry, that’s a software issue, you’re on your own”. I used mine to get the ticket started, but I had resolved the diagnostic phase before we were done creating the ticket and I had figured out it was an .HTACCESS issue which they don’t help with anyway.
Bypass WP-CRON and create a server cron file to do it at the server level. Yep, to me that’s as advanced as it sounds. If you google, you’ll see lots of people who have written why they don’t use WP-CRON, why they turn it off, etc. Almost all of them are hardcore techies who are totally comfortable running LINUX, the type who would thumb their nose at building a Raspberry Pi project as “toys for kids”. If you go this route, and you have access to CPANEL setup, it really is a single line to tell it what to run and when, but I confess it didn’t work for me, and I didn’t want to mess around too much in that level of direct intervention
You might note that I’ve also left out the nuclear option of reinstalling WordPress from scratch. Honestly, that is pretty low on the list of probable solutions if you have done an upgrade from a previously working version and haven’t mucked around with the configuration too much recently. It *should* work, and if it doesn’t, something else is likely configured wrong, not your basic WP install. Always worth a shot of course, and many people would say to start with that in the first place. Who would say to do that? Those who did everything else, failed, tried a fresh install and it solved their problem and now they preach purity of install. 🙂
So what does this have to do with running my own server?
All of these problems? And I’m only talking about a SMALL software config issue that sent me way down the rabbit hole. If I was running my own server, I would have another 20 possible causes, all having to do with either server configuration or potentially even a hardware issue. One of the “saving” graces to paying someone else to carry the load of managing / running a server with a large hoster is that the cost/benefit ratio to me is WAY in my favour, particularly the number of times a problem like this has frustrated the hell out of me. I don’t need that added stress in my life. And if others are running the same install just fine on the big server, I know it is MY problem, not a technical screw-up that I did in the server setup. Worth it’s weight in gold.
As for me, now that I know it’s an HTACCESS challenge, I can push it to the back burner. I can work around it for now, and at some point, I’ll fix it. For now, I’m leaving it alone.
If you ended up here while searching for a solution, feel free to let me know how it’s going in the comments!
It’s been an interesting six weeks since I started putting some serious time into the website. I’ve played with a bunch of content management systems, and some desktop software packages that design sites out of the box, etc. I even had DRUPAL working pretty well. But there were a few problems with each, and for DRUPAL, it was getting my photo gallery to work.
I mentioned last time it was the bane of my existence to get it up and running, and I reconfigured that sucker ten ways from Sunday. Nada. Don’t get me wrong, I doubt it’s DRUPAL’s fault. It is more that it is a step above my needs and yet a few of the simpler CMS software bundles didn’t handle a few key features.
So, in the end, I’ve gone with a mish-mash of software that my host provides automatically pre-configured out of the box. For those of you familiar with Photo Gallery 2, I’ve got my pictures site going under that. For the most part, I’m pretty happy with it. I’ve found a general theme that I can tweak enough to live with, upgraded some of the modules and plug-ins, and uploaded a batch of photos.
For my general “blog” area, I’m going to go with WordPress for now. It seems to be fairly robust and not TOO complicated (I hope). For the rest of my “static pages”, I think I’m going to go with MediaWiki.