Tag Archives: CakePHP

Sendmail not working? Maybe your server’s IP is on a block list

This is pretty arcane, even for me, but since I spent several hours troubleshooting this problem this week — and the solution was nowhere to be found on Google — I figured it was worth sharing.

My CMS, cms34, as I’ve mentioned a few times before, is built on CakePHP. Some features of cms34 include automatically generated email messages. CakePHP has a nice email component that facilitates a lot of this work. It can be configured to use an SMTP server, but by default it sends mail directly from the web server using whatever you have installed on the server, either the ubiquitous sendmail or the more powerful (and capitalized) Postfix. Don’t unleash a deluge of flame comments on me, but I’m using sendmail. So be it.

All was working well until a few weeks ago, when suddenly none of the mails were being sent. There were no errors on the website; the messages just wouldn’t go through. What was more confusing was that messages being sent to my own domain did go through, but for those being sent to my clients’ domains, nothing.

Nothing except log entries, that is. Specifically, the mail log was filling up with lines like this:

Sep 13 13:45:56 redacted sm-mta[28158]: o8DIjsx0028156:
to=<redacted@redacted.com>, ctladdr=<redacted@redacted.com>
(33/33), delay=00:00:02, xdelay=00:00:01,
mailer=esmtp, pri=120799, relay=redacted.com.
[123.456.789.000], dsn=5.7.1, stat=Service unavailable

(Note that I’ve removed the real email and IP addresses to protect the innocent, namely myself.)

“Service unavailable,” huh? I researched that error extensively, without finding much. Eventually I was led to believe it may be an issue with my hostname, hosts, hosts.allow and/or hosts.deny files.

A few relevant points: 1) my hosts.allow file only contains one (uncommented) line: sendmail: LOCAL and 2) likewise, the hosts.deny file only contains: ALL: PARANOID. I’ll save you some time right here: the problem I had ended up having nothing whatsoever to do with any of these host-related files. Leave ’em alone.

After following a number of these dead ends, I was inspired to check the mail file on the server for the user Apache runs as, in my case www-data. On Ubuntu Linux (and probably other flavors), these mail files can be found in /var/mail. Indeed, there were some interesting things to be found there, namely, a number of references to this URL:

http://www.spamhaus.org/query/bl?ip=123.456.789.000

(Again, the IP address has been changed… and yes, I know that’s not a valid IP address. That’s the point.)

I was not previously aware of The Spamhaus Project, but perhaps I should have been. The reason my messages weren’t getting through was because my server’s IP address was on the PBL: Policy Block List. Essentially, that is a list of all of the IP addresses (or IP ranges) in the world that, according to a well-defined set of rules, have no business acting like SMTP (Simple Mail Transfer Protocol*) servers — the servers that send mail out.

It stands to reason that my server was on this list; technically it’s not an SMTP server. But it’s perfectly legitimate for a web server to be running sendmail or Postfix or something of that nature, and sending messages out from the web applications it runs. Fortunately, it’s easy to get legitimate servers removed from the PBL. Simply fill out a form, verify your identity (via a code sent to you in an email message), and within about an hour, the changes will propagate worldwide.

Success! So if you’re in the same kind of situation I was in, where everything seems to be configured properly but your messages just aren’t going out for some reason, try checking Spamhaus to see if your IP is on the PBL.

* If you made it this far in the post, I shouldn’t have to explain the acronym. But I will anyway, as is my wont.

Making CakePHP’s TreeBehavior work with scope

We’re not talking mouthwash here. We’re talking code.

CakePHP’s TreeBehavior is cool, but tree traversal is a pretty arcane concept, even for developers, and MPTT is not something that is easy to digest mentally, or to develop around. Unfortunately, it’s also not incredibly efficient on large data sets. Even not-so-large data sets, on the order of a few hundred items, make certain actions — reordering, in particular — really processor intensive.

I’m using a JavaScript drag-and-drop tool in cms34 to allow admins to manage the page tree on their sites, and the data gets stored using CakePHP’s TreeBehavior, which is MPTT-based. The problem is, it really wasn’t working. So I completely rebuilt the code that assigns the über-critical “left” and “right” values to each node in the tree. My new way is much faster, even though it may break a few rules, and requires bypassing TreeBehavior’s callbacks.

Once I got that working, I was delighted, but then I discovered some other problems, namely pertaining to… scope. My CMS supports multiple sites in one installation, which means multiple trees, which means scope. The problem is, I was having a hell of a time figuring out just how to make scope work with TreeBehavior. Finally I found a link to a succinct and effective solution.

The upshot here is that you can’t define your $actsAs in the model, because… well… to be honest, I only have a vague understanding of why not, but essentially it’s due to the split roles of the model-view-controller framework. You’re building a rule that requires access to specific data values, which is something that needs to happen in the controller; the model is strictly for the abstract structure of the data. I understand it just enough to agree that it makes sense not to do it in the model. Which means you have to do it in the controller. The sample code from the link above goes a little something like this:

function add() {
    $this->Task->Behaviors->attach('Tree', array(
         'scope' => "Task.schedule_id = {$this->data['Task']['schedule_id']}"
    ));
    $this->Task->create();
    $this->Task->save($this->data);
}

That didn’t quite work in my situation, but it got me far enough along that I could figure out what to do from there.

I still need to do a little more testing to make sure my solution to the more efficient tree reordering is rock solid, and then I’ll post a tutorial. But for now, I hope this helps spread the word that scope does work on TreeBehavior… if you do it right.

Update (September 22, 2010): Although this didn’t really seem to be breaking anything, just throwing up a warning (when debugging was turned on), I discovered a minor issue with this code yesterday. Turns out CakePHP expects the value of scope to be an array. Just taking the string it was defined as and wrapping it in array() did the trick.

CakePHP and the Session.timeout SNAFU

As I’ve mentioned before, my CMS, cms34, is built on the CakePHP framework. I love CakePHP, but it’s not perfect.

A client contacted me last week, reporting that customers using her website — specifically, a complex form that might take 30 or more minutes to complete — were having their sessions timeout during that process. Now, yes, you may point out (and you would be right) that this is an excellent scenario for using AJAX to save a draft of the users’ form data, or at least ping their session to keep it alive. And you would be correct. But there are two reasons I did not take this path in response to the problem: 1) I’m trying to remember to follow the web standards principle that JavaScript should be used only to enhance functionality, but the site should still work even with it turned off (though in practice I often forget to honor this principle), and 2) the site is in its peak of activity this week, and it would be daft to implement a major functionality change in the form right now — not enough time for adequate testing.

So, the task at hand was simply to increase the session timeout period, so users could stay on the form for a longer time without losing their sessions.

Seems easy enough. The CakePHP config.php file has some settings that play directly into this: Session.timeout and Security.level.

Session.timeout is an integer representing the number of seconds in the session. Well, not exactly. It’s a multiplier value, and the actual session timeout duration is determined by multiplying Session.timeout by the value associated with Security.level, which itself has three possible values: high, medium and low, corresponding respectively to ×10, ×100 and ×300.

I already had Security.level set to low and Session.timeout set to 1800, so really, the sessions should have stayed alive for over six days of inactivity. And yet, sessions were timing out after about 20 minutes. What gives?

What gives is another value in the config.php file: Session.save.

CakePHP offers a number of ways to store session data, but the default value is php, which means it uses the built-in session functionality in PHP. The problem is, PHP has its own session timeout setting (in the php.ini file), and frankly my dear, PHP doesn’t give a damn what CakePHP’s session timeout setting is. And so, with Session.save left at its default php value, my sessions were timing out after 20 minutes (the PHP default) no matter what I changed in config.php.

So, an experiment: I decided to change Session.save to cake, which tells it to use CakePHP’s own session cache, located inside your CakePHP application under app/tmp/sessions. (Be sure that path is fully writable by Apache!)

Success! I left a browser window open on my test site overnight, and the session was still active the next morning. But along the way in my troubleshooting, I had changed my Session.timeout value to 14400, which when combined with the Security.level setting of low (that is, when multiplied by 300), had my sessions enduring for a ridiculously long 50 days! Some quick work with a calculator (although, in retrospect, I ought to have been able to divide 14400 by 50 in my head) told me to set Session.timeout to 288 if I wanted my sessions to last for 24 hours.

Migrating from CakePHP 1.2 to 1.3: My Story (Part One of… Possibly More than One)

For the past couple of years I’ve been working on a rapidly evolving CMS project called, not-so-creatively, cms34. It’s built on the open source CakePHP framework.

I love CakePHP. I’ve dabbled with a couple of other frameworks (notably, Zend Framework), and while CakePHP certainly isn’t perfect, I’ve found it by far the easiest to jump into and quickly get a powerful and reliable web application running.

When I started cms34 in late 2008, it was using CakePHP 1.2.0. Since then I’ve kept up with most of the minor point releases, upgrading as far as the current version, 1.2.7, as easily as simply swapping out the cake directory. And in that time, cms34 itself has gone through several major and minor revisions, currently up to what I’m calling version 3.2.

In preparation for version 3.3, I’ve decided to take the plunge and upgrade from CakePHP 1.2.x to the newly released CakePHP 1.3.0. Version 1.3 offers a number of improvements, but it also includes the kinds of changes typical of a major point release that mean an upgrade is not such a simple task. I’ve run into a few issues with the migration from 1.2.x to 1.3.x, many of which are not covered in the project’s official migration documentation, partly due to the incompleteness of the documentation, and partly due to some idiosyncrasies of my application that I wouldn’t expect the documentation to cover. But since I’m probably not in this boat alone, I thought I’d share some of the problems I encountered, and the solutions I found.

Getting started: RTFM

First things first. Download CakePHP 1.3.0. Be sure to get the release version (the one in the list with no hyphenated qualifiers after “1.3.0”). Unzip it and then follow the migration instructions to get started. For the most part this means swapping out the cake directory along with a couple of other files, and then going through all of the notes and making any necessary changes to your application.

In my own experience, I didn’t have to change much: I’ve been trying to keep up with the recommendations in the version 1.2 docs, so I haven’t been using any functionality that got dropped or deprecated. The writing has been on the wall for some time with most of these features and unless your application started with version 1.1 or 1.0, you’re probably not using any of these dead methods anyway.

A few specific issues I did need to address:

  1. Unlike with minor 1.2.x updates, there are a few files outside of the cake directory that need to be replaced. I found it best to just go through the entire release package and replace all files (except those I had modified) with their new versions.
  2. Update your config/core.php file, using the new version as a model. Mine had lots of changes, and I had stripped out all of the instructional comments to make the file less cumbersome to deal with. I wanted to keep those comments stripped out, so I just updated my file with what I needed, and then I kept the distribution’s version of the file in config as core.dist.php so I can refer back to it in the future if I run into any issues.
  3. There are some mixed messages in the documentation regarding the deprecation of AjaxHelper: the migration appendix says it’s deprecated but the main documentation doesn’t indicate that. I’m using it a fair amount and don’t want to have to rework all of that code, so for now I’m leaving it alone.
  4. JavascriptHelper is another matter, but since it appears there are perfect replacements for its functions now in HtmlHelper, I just did a straight-up find-and-replace, changing all instances of $javascript->link() to $html->script() and $javascript->codeBlock() to $html->scriptBlock(). I should note that so far I haven’t really tested any of this functionality to ensure that it’s still working, but no obvious errors have cropped up.

But wait, there’s more!

After following the migration docs, I was overly optimistic that things would just work. They didn’t. First up, I encountered this:

Notice (8): Undefined property: PagesController::$Session
[APP/controllers/app_controller.php, line 383]

Your line number will probably be different, but if you’re using sessions at all (and why wouldn’t you be?), you’ll get this error somewhere. It’s not mentioned in the migration docs, but I found out here that the Session component and helper are not instantiated automatically anymore. The solution is quite easy though. In app_controller.php just make sure that you add 'Session' to the arrays defined for var $components and var $helpers.

That was the biggest and most obvious problem I encountered. Once I had fixed that, I could at least get (some) pages to load, albeit with a few other issues. Namely, my theme was not being applied to the site. I’ll get to that in a minute though, because the problem I decided to fix next came in my CMS admin interface.

Pagination (even if you don’t want it)

I use the PaginatorHelper a lot, especially in the CMS admin. My CMS is modular: there’s a “module” (my term for any model-view-controller group) for pages, a module for blog posts, a module for uploaded files, a module for project porfolios, a module for the event calendar, etc. Each module is represented in the admin interface by a navigation tab, which when clicked takes the user to an index page with a paginated, tabular list of all of the records in the database for that module. In practice (on my own site — yes, I use my CMS to run my own site, of course!), it looks like this:

Unfortunately, the pagination was mysteriously hosed, with the following error:

Unsupported operand types in CAKE/libs/view/helpers/paginator.php
on line 616

What the…? Well, as it turns out, the methods $paginator->prev(), $paginator->next() and $paginator->numbers() have had changes to their input parameters that are currently not very well documented, in that the documentation doesn’t mention that they exist. They are mentioned in the API, but as usually happens when I refer to the API, I found it wanting, to the point that I resorted to digging into the source code directly to try to make sense of what these methods actually do.

In short, where you could previously pass these methods a URL as a string (as the second parameter for prev() and next() and the first for numbers()), now they expect arrays. And numbers(), in particular, really doesn’t like that parameter to be a string (because at one point it uses the += operator to merge two arrays, one of which is the first input parameter), resulting in the Unsupported operand type error.

Fortunately, in my case at least, I found that I didn’t need to pass the URL into any of these methods at all, and that just removing them altogether from the calls fixed the problem.

There was an ancillary problem once I got past the PHP error: inactive “Previous” and “Next” links were appearing on the page when there was no pagination. This was an unwanted addition, but I quickly realized this was an intentional change, and is well-documented. I was able quite easily to make them go away with a little CSS:

span.prev, span.next { display: none; }

With that minor crisis resolved, I could move back to the matter of the theme not showing up.

Changing themes

First, some background on my CMS: I’ve built the application so that I can have a single “core” system that works for numerous client websites, organized in a way that it’s easy to deploy updates across a number of separate sites. The system is also built to support multiple sites running on a single installation (by way of a Site model, with every piece of data in every other model having a site_id field keying the content to the proper site).

There are two key features of CakePHP that I rely on to make this all work: the look-and-feel (and site-specific view functionality) of each site is managed using Themes, and any site-specific controllers and views are handled with Plugins.

It is documented, unfortunately not in the migration appendix, that version 1.3 of CakePHP has changed how theme assets (static files like images, JavaScript and CSS) are organized. Now, instead of placing these in a directory under app/webroot/themed, you create a webroot directory inside the appropriate theme folder in app/views/themed. This is nice, in that now all of the files for a theme are in one location. It can cause a performance hit though, as these files are now being processed through PHP. You can still put the files under app/webroot to avoid this performance hit, but, annoyingly, the path has changed, as noted in the documentation. I decided to go with the new approach under app/views, as performance is not a major issue with the current implementations of my CMS, and the benefit of a single, consolidated directory for each theme is worth the trade-off. If performance does become an issue, though, at least now I know what to do about it.

More to come…?

After completing the above changes, I spent some time exploring my test application, both the user-facing site and the CMS admin interface, and so far I haven’t come across any other problems. Which is not to say I’m ready to roll this out to all of my clients tomorrow morning. I want to make the switch soon, as maintaining two separate code bases is a cumbersome task for a solo freelancer. But while things look good for now, this kind of major upgrade requires some solid testing and troubleshooting. If I run across any other issues, I’ll post them in Part Two.

TinyMCE and the relative URLs SNAFU

My CMS, cms34, uses a few third-party tools for certain complex features. One of the most useful is TinyMCE, a JavaScript/DOM-based drop-in WYSIWYG text editor. If that doesn’t mean anything to you, you probably want to stop here, but in case you’re a glutton for punishment, it is, in short, a way to produce HTML-formatted text with a word processor-like interface. Think Microsoft Word in a web browser, with the results being formatted for display on a web page. Slick.

Anyway, there’s been one nagging problem: When users paste in URLs for links, TinyMCE converts any on-site links into “relative URLs” — it strips out the domain name. This is not necessarily bad; in fact, for the most part I would want it to do that. But for some reason the nature of my CakePHP-based CMS seems to confound TinyMCE’s ability to properly determine the relative URL. And what’s worse, the CMS includes an enewsletter editor which has to have absolute URLs, but TinyMCE was converting them to relative URLs even if the user pasted in an absolute URL.

A little research led me to a handy explanation in the TinyMCE Wiki. Basically, if you want your URLs to always be absolute, make sure your TinyMCE configuration includes the following:

relative_urls : false,
remove_script_host : false,
document_base_url : “http://www.site.com/path1/”

Of course, you’ll want to change the value of document_base_url to be the actual base URL of your website. As it happens, my CMS has a global JavaScript file that creates a variable called baseUrl that I can use anywhere in JavaScript to substitute for the full base URL of the website. So, in my case, I set the value for document_base_url equal to baseUrl.

And, voilà, it seems to work!