jwills.co.uk website update

Have been updating this site a little bit. I launched jwills.co.uk a short while ago as a way to connect things I work on in one place, and act as a bit of a portfolio site. I also had an old Magento blog, and a small website for the O2 Joggler which I wanted to manage in a single place.

So what’s new? Well, I have now switched the theme (off the shelf WordPress theme with minor tweaks – sorry!), and have added the following sections to Projects:

These are all projects I work on, or have contributed to. I also automatically pull in the news feeds from XBMC4Xbox and ExoticA to this site.

In other news, I have been busy on RetroPie getting it ready for the final 3.0 release, which I hope will be ready v.soon – there is an issue with a single emulator I want to look at, and then I’m happy to push out a new image.

Magento Connect/Pear channel not updated ?

We manage our Magento shops in source control, but upgrade extensions as needed via Pear and Magento’s channels. However since mid December 2009, no upgrades have been showing up. Looking at the pages on the Magento site, there are upgrades available for some of the extensions we are running, but they are no longer being shown. I wonder if some meta data for the pear channel is not being updated correctly?


$ ./pear list-upgrades
Channel connect.magentocommerce.com/community: No upgrades available
Channel connect.magentocommerce.com/core: No upgrades available

I am not the only one with this problem, as someone posted to the Magento Connect forum (http://www.magentocommerce.com/boards/viewthread/73424/). This isn’t my normal kind of post, but I’m hoping someone with the power to fix this will notice.

Sorry for the lack of posts here. I have not been working with Magento that much in recent months. I do have some other short articles  planned though. If you need any development work feel free to contact me (jools [at] oxfordinspire.co.uk).

Setting default options for configurable products

We are using the configurable products feature of Magento to group together some of our products that differ for example only by colour or number of items in a pack. In our case there is just a single “option” for the product and we wanted it to default to the “first” option in the dropdown, rather than the Magento default of forcing the user to choose an option.

As the dropdown for a configurable product is filled in on the client side with javascript this required some javascript code to “switch” to the first option and then reload the price. Here is the javascript we used to do this . This is a diff file to be applied against the /design/frontend/default/default/template/catalog/product/view/type/options/configurable.phtml file. This path will differ when you are using your own skin.

Index: view/type/options/configurable.phtml
===================================================================
--- view/type/options/configurable.phtml	(revision 73)
+++ view/type/options/configurable.phtml	(revision 74)
@@ -42,5 +42,10 @@
     </dl>
     <script type="text/javascript">
         var spConfig = new Product.Config(<?php echo $this->getJsonConfig() ?>);
+        // set the defaults to the first option
+        for(var i=spConfig.settings.length-1;i>=0;i--) {
+          spConfig.settings[i].selectedIndex = 1;
+        }
+        spConfig.reloadPrice();
     </script>
 <?php endif;?>

*Update*

Unfortunately the code doesn’t work as expected on IE due to http://support.microsoft.com/kb/927917 “Message: HTML Parsing Error: Unable to modify the parent container element before the child element is closed (KB927917)”.  My fix was to just disable this for now for IE by wrapping the changes in

if ( ! Prototype.Browser.IE ) {
...
}

If anyone knows a workaround for this, please let me know! Did I mention how much I hate IE? !

Creating a simple news feed module (for display within static blocks)

Here I will explain how I implemented a simple news feed feature that can be used inside static blocks. Before creating this, I had a look, and only found a commercial offering for this, so I hope this is of use to other people. It also was a learning experience for me, as it is the first code I have done for Magento that utilises a custom template.

To save myself a little work, I took an off the shelf RSS and Atom feed parser called “Simplepie”. This might not have been the best choice as there are a couple of issues with it. Firstly it is reasonably large, and no doubt does a lot more than I require. Secondly, I had some problems getting it to work with Magento. This was due to simplepie having php4 compatibility, and because I like to run my php5 with E_STRICT error reporting, php complained. This however was a quick fix and I will cover that later. If anyone knows a modern, compact php5 rss/atom parser that is any good, please do let me know.

First let’s start then with simplepie, the modifications to make to it, and where to  install it. Paths referenced in these instructions should be considered relative to the Magento root folder.

Download the latest simplepie archive from the website (http://simplepie.org/). At the time of writing this is version 1.1.3. Unpack it somewhere. Create a folder in your Magento installation called lib/simplepie.

Copy the simplepie.inc and the idn folder (and any LICENCE.txt etc) to the magento/lib/simplepie folder. You should now have a folder structure similar to

lib/simplepie/
              LICENSE.txt
              simplepie.inc
              idn/
                  idna_convert.class.php
                  LICENCE
                  npdata.ser
                  ReadMe.txt

Now we will make some simple modifications to the simplepie.inc file (and also rename it to simplepie.class.php – this is just my own preference). Simplepie includes php4 compatibility that could cause problems running on php5 depending on your error reporting settings. The following sed commandline will correct this so simplepie will not cause errors and will ingrate with Magento on a PHP5 setup with E_STRICT error reporting.

sed -e "s/& new/ new/g" simplepie.inc >simplepie.class.php

After running this, you can remove the old simplepie.inc. There are some other things within the simplepie code that will also cause problems running in E_STRICT mode on php5, but we can work around this by changing the error reporting level in our module. I will explain this further on.

Our newsfeed module will be stored in app/code/local/Maglife/Newsfeeds. The Maglife folder is our module namespace, and the Newsfeeds is the name of the module. The structure here is as follows

app/code/local/Maglife/Newsfeeds/
                                 Block/
                                       View.php
                                 etc/
                                     config.xml

The file View.php contains the main module code, and the config.xml contains the module configuration, including version number, and class name.

File: app/code/local/Maglife/Newsfeeds/etc/config.xml
This file contains the module configuration including module version number and our module class.

<?xml version="1.0" encoding="UTF-8"?>
<config>
    <modules>
        <Maglife_Newsfeeds>
            <version>0.1.0</version>
        </Maglife_Newsfeeds>
    </modules>
    <global>
        <blocks>
            <Maglife_Newsfeeds>
                <class>Maglife_Newsfeeds_Block</class>
            </Maglife_Newsfeeds>
        </blocks>
    </global>
</config>

File: app/code/local/Maglife/Newsfeeds/Block/View.php

This file contain the main code for our module. Our module will also have a template file, that also contains some code, but the bulk of the work is done here, so that the template is kept as simple as possible.

Note that we include the idna_convert.class.php first, as otherwise there are issues with the class being autoloaded from the wrong location. As I mentioned earlier a more lightweight parser may well be better suited, or perhaps using simple_xml to parse the feed manually. You will also see in the class contructor we switch the php error reporting and then back in the destructor. This is to stop issues with simplepie and E_STRICT mode on php5.

This code is very simple, and could be extended further. When we utilise the Newsfeeds module from magento, we will pass in parameters for the feed url, the number of items to display and an optional title. The initFeed() call will set up and retrieve the feed from the feedurl parameter configured within our static block (covered later on).  I have three main calls. These are initFeed() that will be called from the start of the Newsfeed template (listed later),  a call getFeedCount to get the number of feed items to display (or return a default of 5), and getFeedTitle that will return the title of the feed (or a pre configured title passed from the static block). The initFeed call has some code to configure the cache folder. The Mage::getConfig()->createDirIfNotExists($cache_dir); call is used to create a folder in the magento/var/cache folder called “newsfeeds”. This relies on the /var/cache folder being writable by the web server process. This however should already be setup correctly on your magento installation.

<?php

require_once('simplepie/idn/idna_convert.class.php');
require_once('simplepie/simplepie.class.php');

class Maglife_Newsfeeds_Block_View extends Mage_Core_Block_Template {

    public $_feed;
    private $_old_reporting;

    public function __construct()
    {
        // simplepie is made for php4 and will report errors in E_STRICT mode
        $this->_error_reporting = error_reporting( error_reporting() & ~ E_STRICT );
        $this->_feed = new SimplePie();
    }

    public function __destruct()
    {
        // restore error reporting to previous setting
        error_reporting($this->_old_reporting);
    }

    public function initFeed()
    {
        $this->_feed->set_feed_url($this->getfeedurl());
        $var_dir = Mage::getConfig()->getTempVarDir();
        $cache_dir = $var_dir . '/cache/newsfeeds';
        Mage::getConfig()->createDirIfNotExists($cache_dir);
        $this->_feed->set_cache_location($cache_dir);
        $this->_feed->init();
        $this->_feed->handle_content_type();
    }

    public function getFeedCount()
    {
        // if there is a count attribute for our block, use that,
        // alternatively, use a default
        if ( $this->getcount() ) {
            return $this->getcount();
        } else {
            return 5;
        }
    }

    public function getFeedTitle()
    {
        // if there is a title attribute for our block, use that for the
        // heading, alternatively, use the title of the news feed
        if ( $this->gettitle() ) {
            return $this->gettitle();
        } else {
            return $this->_feed->get_title();
        }

    }

}

Now for the template that controls the display/look of the newsfeeds block. You may well want to change this. The template lives in app/design/frontend/interfacename/theme/template/newsfeeds. For the default interface and theme that Magento ships with this would be app/design/frontend/default/default/template/newsfeeds. If you are using a custom theme you will want to create the newsfeeds folder in the relevent theme/templates folder. You are not forced to use this template file and folder name, but if you change them, make sure you reference the template correctly from the static block (shown later). The design and classes used in this template fit well with the Magento “blank” theme. The Magento blank theme is a simple theme that is a good starting point for creation of your own.

File: app/design/frontend/default/default/template/newsfeeds/view.phtml
A simple piece of html and php. The first line calls our initFeed() function that retrieves the feed. We then get the title, and loop through the feed items (displaying them in an unnumbered list). The $this->_feed object is a feed object returned by simplepie, so the code below for getting the items, is the same as you would find in the simplepie documentation.

<?php $this->initFeed() ?>
<div class="block block-newsfeeds">
    <div class="block-title">
        <h2><?php echo $this->getFeedTitle() ?></h2>
    </div>
    <div class="block-content">
        <ul>
        <?php foreach ($this->_feed->get_items(0, $this->getFeedCount()) as $item) { ?>
            <li>
                <a href="<?php echo $item->get_permalink(); ?>"><?php echo $item->get_title() ?></a>
            </li>
        <?php } ?>
        </ul>
    </div>
</div>

This is the last of the files for our module. The only thing left to do, is to activate the module within Magento, and then utilise our module within a static block. To enable our module we must make a small configuration file in app/etc/modules

File: app/etc/modules/Maglife_All.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Maglife_Newsfeeds>
            <active>true</active>
            <codePool>local</codePool>
        </Maglife_Newsfeeds>
    </modules>
</config>

If you already have a Maglife_All.xml file (for example if you have used one of our earlier modules), then just add the <Maglife_Newsfeeds> section to the <modules> section of the xml.

Now, we are ready to try out the module.

Log into your Magento administration panel and navigate to the CMS/Static Blocks menu, and Add new Block. Enter the following information

  • Block Title: Newsfeeds
  • Identifier: newsfeeds
  • Store View: All Stores
  • Status: Enabled
  • Content: {{block type=”Maglife_Newsfeeds/View” template=”newsfeeds/view.phtml” feedurl=”http://news.google.com/news?ned=us&topic=h&output=atom” count=”5″}}

This tells Magento to use our custom block Maglife_Newsfeeds/View, with the template newsfeeds/view.phtml (or another template of your choice), along with the feedurl (for Google news – top stories). The count parameter is the number of items to display. There is also a title parameter that can be set to change the title in the displayed news block. If omitted as above it will use the title returned by the newsfeed.

Save this static block. We now want to make it display on the site. In this example we will make it display at the top right of the homepage. Navigate to CMS/Manage Pages on the administration panel and choose the Home page page that comes with the default magento install. Select the tab custom design. There should be a block of commented out XML. From here you can add layout XML to the homepage. Add the following XML to the bottom in the Layout Update XML textbox.

<reference name="right">
    <block type="cms/block" name="newsfeeds" before="-">
        <action method="setBlockId"><block_id>newsfeeds</block_id></action>
    </block>
</reference>

This tells Magento to display our static block at the top of the “right” section in the layout.

Now, hopefully, you should be able to navigate to your Magento home page and see a list of news items from Google news!

If you don’t I suggest turning on the debug logging from the Magento System/Configuration menu and checking for debug output  in var/logs/exception.log.

Please do let me know if you find any mistake/typos in this article. It is possible I may have forgotten a step or two. I hope this is of use to you, and at least gives some more ideas on how to work with Magento.

Running multiple Magento stores on multiple domains from a single index.php

For our eCommerce project, we are building multiple websites each with multiple languages, but all managed from one Magento install. We wanted to run each website from it’s own domain, with each store view as a subfolder for the language, such as (The base link urls are configured like this for each store view from the Magento admin panel):

  • http://store1.example.com/en/ (store name store1_en)
  • http://store1.example.com/fr/ (store name store1_fr)
  • http://store1.example.com/de/ (store name store1_de)
  • http://store1.example.com/es/ (store name store1_es)
  • http://store1.example.com/nl/ (store name store1_nl)
  • http://store2.example.com/en/ (store name store2_en)
  • http://store2.example.com/fr/ (store name store2_fr)
  • http://store2.example.com/de/ (store name store2_de)
  • http://store2.example.com/es/ (store name store2_es)
  • http://store2.example.com/nl/ (store name store2_nl)

Looking around the net for solutions, it seems the majority of suggestions are to duplicate the magento index.php and .htaccess into a folder for each language/store view. This would mean more files to maintain, and a more complicated structure. We also did not want to have to rely on .htaccess for the security and url rewriting, but rather maintain the rewrite configuration as well as the store URL to store name mapping in the main Apache config. We also didn’t want to have to extend the folder structure just because we added a new store view/language in the future.

I came up with a solution for this, utilising a slightly modified index.php which “runs” each store, does some querystring modifications, and an Apache configuration to handle the rewriting. This allows us to easily add another store view, just by altering the Apache config.

First of all here is our site structure. the js/media/skin folders are symbolic links to the parent magento folders (that is outside of the web accessible root)

site/
    magento/ (standard magento install)
    store/
          index.php
          js     -> ../magento/js
          media  -> ../magento/media
          skin   -> ../magento/skin
          report -> ../magento/report

Each virtualhost points to the store/ folder. It seemed safer to keep the main magento folder outside of the site root, and everything so far seems to work ok like this. Note there is no .htaccess. We will do the rewriting from the main Apache config.

Here is the example Apache config. I have split it into two files in this example. We run on a Debian system, and Apache has a layout with a conf.d and a sites-available/sites-enabled. We are also using mod_macro for the configuration, as this allows us to reduce configuration duplication.

File: /etc/apache2/conf.d/mageno

<Macro MagentoVHost $store $langs>
  Options +FollowSymlinks
  DocumentRoot /path/to/site/store
  SetEnv MAGENTO_STORE $store
  # For information on the next variable, see earlier blog post.
  # SetEnv MAGENTO_DEVURLS /path/to/urls.ini
  RewriteEngine On
  # we always have English as our default language
  RedirectMatch ^/$ /en/
  RewriteRule ^/($langs)/(.*)$ /index.php/$2 [E=MAGENTO_LANG:$1,L]
</Macro>

<Directory /path/to/site/store>
  RewriteEngine On
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_FILENAME} !-l
  RewriteRule ^(.*)$ /index.php/$2 [L]
</Directory>

File: /etc/apache2/sites-available/store1.example.com

<VirtualHost *:80>
  ServerName store1.example.com
  # this store has 5 languages (store views)
  Use MagentoVHost store1 en|fr|es|de|nl
</VirtualHost>

<IfModule mod_ssl.c>
  <VirtualHost *:443>
    ServerName store1.example.com    # this store has 5 languages (store views)
    Use MagentoVHost store1 en|fr|es|de|nl

    SSLEngine on
    SSLCertificateFile /etc/apache2/ssl/your_certificate.pem

  </VirtualHost>
</IfModule>

Now a brief explanation on how the above works. All stores use a language in the url. If you visit the root,  the

RedirectMatch ^/$ /en/

line will redirect to the default store view (English). The virtualhost example above passes the store name to our macro. The macro then sets the variable MAGENTO_STORE to this. Depending on the language selected (from the chosen url) Apache will set an environment variable MAGENTO_LANG. So if we visited http://store1.example.com/de/ MAGENTO_STORE would be set to “store1” and MAGENTO_LANG would be set to “de”. Now we just need to process these in the main index.php that “Runs” the Magento store of our choice.

File: site/store/index.php
The following code replaced the last two lines in the default magento config

umask(0);
Mage::run();

# we use the url structure as /en/blah.html /fr/blah.html for each language
# language is set by mod_rewrite from the apache config
$lang = 'en';
if ( isset($_SERVER['MAGENTO_LANG']) ) {
  $lang = $_SERVER['MAGENTO_LANG'];
}

# store is set from the apache configuration depending on domain.
# this allows us to configure stores without having to modify multiple index.php files etc.
$store = '';
if ( isset($_SERVER['MAGENTO_STORE']) ) {
  $store = $_SERVER['MAGENTO_STORE'];
}
$store .= '_' . $lang;

# replace the REQUEST_URI such as /en/blah.html with /blah.html
$_SERVER['REQUEST_URI'] = preg_replace(&quot;#^/$lang(/.*)#i&quot;,'$1',$_SERVER['REQUEST_URI']);

# we want our /lang/ urls to take priority over store querystrings
# but not if we are in the admin panel
if ( ! preg_match('#^/index.php/admin/#i',$_SERVER['REQUEST_URI']) ) {
  $_GET['___store'] = $store;
}

umask(0);
Mage::run($store,'store');

The comments in the code should explain how it works. We construct our store name from the store name and language. In our example this would be store1_de. The language is then stripped from the REQUEST_URI environment variable, as this breaks magento’s search engine URL processing. (So even though we are at the URL http://store1.example.com/de/somepage magento thinks we are at http://store1.example.com/somepage. Also note the code to override the ___store querystring parameter. We want our url path to specify which store we are in over any querystring magento chooses. This solves an issue I had where Magento would think we are in the wrong store. Perhaps due to the “current” store being stored in a cookie or a querystring parameter being appended by Magento. This parameter is also used on the administration side, so we don’t want to interfere with if we are in the admin panel. Finally we “run” the store of our choice.

Hopefully this has made sense. It might look rather complicated, but does allow for a flexible setup. If we wanted to add a new store view (language) to our example store1, we can simply add it from the magento admin panel, and modify the apache config. The index.php does not need to change.

Magento Base URLs and dev/staging installations

One of the earlier problems I came across when setting up Magento, was that the Base URLs for the websites/stores are stored in the database. We wanted a scenario where we can have a staging site, and a development site, and transfer the database between them. With the way Magento works, this would require updating the Base URLs each time, and means that we couldn’t easily share a single database between a local development copy and a shared development copy running from our svn.

My first idea to workaround this was to create a small tool, that updated the Base URLs stored in the database, however this didn’t solve the issue of sharing one database between different installs. I looked into the Magento code, and saw that there is a function Mage::getBaseUrl (which in turn calls Mage_Core_Model_Store->getBaseUrl). If I could override this function on the development copies, I could replace the live Base URLs stored in the database with whatever local address we were using.

One of the best places to find development information seems to be on other people’s blogs, and this was one of the reasons why we thought it a good idea to blog our development here. I found http://www.belanur.de/blog/2008/09/05/overriding-core-models-with-a-custom-module/ invaluable when it came to learn how to override classes/function in Magento.

On to the code. The code is split into 4 files. Two XML configurations, and the php with the new getBaseUrl call. The other file is a ini config file for specifying how the Base URLs should map to the development/staging URLs.

Assuming you have a working configured Magento install. The following paths are all referenced relative to the main Magento folder.

File: app/etc/modules/Maglife_All.xml
This file activates our module. Magento merges this xml with its own xml to create a config.

<?xml version="1.0"?>
<config>
    <modules>
      <Maglife_Core>
        <active>true</active>
        <codePool>local</codePool>
      </Maglife_Core>
    </modules>
</config>

File: app/code/local/Maglife/Core/etc/config.xml
This file specifies to Magento what the module does. In this case it says the model / class we want to override and points to the replacement file (the line Maglife_Core_Model_Store translates to the path Maglife/Core/Model/Store.php which contains our code). This is at least my current understanding as I haven’t found any comprehensive documentation on the various xml tags yet.

<?xml version="1.0"?>
<config>
    <global>
        <models>
            <core>
                <rewrite>
                    <store>Maglife_Core_Model_Store</store>
                </rewrite>
            </core>
        </models>
    </global>
</config>

File: app/code/local/Maglife/Core/Model/Store.php
This contains the actual code. We call the parent getBaseUrl function, and then if the ENV variable is set, we replace the base url with a url provided in the urls.ini file (passed in the MAGENTO_DEVURLS variable).

<?php

class Maglife_Core_Model_Store extends Mage_Core_Model_Store
{

    public function getBaseUrl($type=self::URL_TYPE_LINK, $secure=null)
    {
        $store_code = $this->getCode();
        $url = parent::getBaseUrl($type, $secure);
        if ($url_ini = @$_SERVER['MAGENTO_DEVURLS'])
        {
            if ($urls = parse_ini_file($url_ini))
            {
                $host = parse_url($url, PHP_URL_HOST);
                if (isset($urls[$host]))
                {
                    $url = str_replace('://'.$host.'/', '://'.$urls[$host].'/', $url);
                }
            }
        }
        return $url;
    }

}

?>

File: urls.ini (can call this anything you like)

livestore.example.com = staging-livestore.example.com
livestore2.example.com = staging-livestore2.example.com

Then in my apache config on the staging/dev site I have the line

SetEnv MAGENTO_DEVURLS /path/to/urls.ini

and that’s it. I can now copy our live database to the staging server for example, without having to manually alter the Base URL config data in the database. I hope this is some use to others.

Magento Blog

We chose the Magento software for a new e-commerce project, and decided it would be a good idea to document our findings and ideas for others to use. Magento documentation is a little thin in places, and we have found a good deal of information from reading other people’s blogs.