Magento 2 Composer Module Update

People often asks us how to do a Magento 2 Composer Module Update for their extensions whenever they want to release a new version of their modules.

There are different ways of doing it, we will show you how to do with with git tags and a semantic versioning strategy for the releases, which we thing is the most professional and reliable way.

We will continue with the example of our previous post where we showed you how to create a hello world extension in Magento 2 with composer.

For the newcomers to composer, et me quickly give you an brief explanation of how does it work, more specifically, how we use it together with Magento 2 while developing new modules:

  • To install a new extension developed by ourselves (or by a third party): We normally use “composer require vendor-name/extension-name” from the root folder of Magento 2. Which, by default (depending on the value we’ve set for “minimum-stability” and “prefer-stable” in our composer.json file) will attempt to install the latest version of that extension that meets its dependencies/requirements within your system/application. This means that, for example, if you are using php 5.5.x and the latest version of the extension that you are installing has a dependency (composer.json “require” section) like:
    "php":">=5.6"

    Then, the latest version of the extension cannot be installed on your system. Composer will firstly try to install a newer version of the dependency, and failing that, will search backwards previous versions of the extension until it finds the most recent version that works with php 5.5. Eventually, it will fail if it cannot find any version installable (that meets all the dependencies) in your system . Normally, you just run the command and everything runs smoothly, composer will take care of everything from you, but sometimes it will fail with an error like below:

    composer require lumbrales-software/magento2-first-module 
    Using version dev-master for lumbrales-software/magento2-first-module     
    ./composer.json has been updated
    Loading composer repositories with package information
    Updating dependencies (including require-dev)                             
    Your requirements could not be resolved to an installable set of packages.
    
      Problem 1
        - magento/project-community-edition dev-master requires magento/product-community-edition 2.0.2 -> no matching package found.
        - magento/project-community-edition 2.0.2 requires magento/product-community-edition 2.0.2 ->  no matching package found.
        - magento/project-community-edition 2.0.1 requires magento/product-community-edition 2.0.1 ->  no matching package found.
        - magento/project-community-edition 2.0.0-rc2 requires magento/product-community-edition 2.0.0-rc2 ->  no matching package found.
    

    This can sometimes require a system update, or like in the above example, adding a custom repository where composer can find the required package.

  • To release a new version (code changes) of your extension: This is the interesting part. People are usually confused and run “composer update” and expect it to do the job, but the problem is that the above command can often lead to undesired and unexpected results, since it will try to update every single package, which is not always (almost never) what we want. Normally, you want to update an specific package, or maybe even a whole vendor, but not everything at once. Therefore, we recommend to tell composer what to do, like in the example of our previous post:
    # Update an specific package
    composer update lumbrales-software/magento2-first-module
    # Update all packages of the vendor
    composer update lumbrales-software/*
    

Having said that, now let’s focus on how to release a new version of our extension.

The process is actually simpler than it might seem, all you have to do is implement your new functionality, code changes, whatsoever, using whichever git branching strategy you want, and once you are ready, you can do the following steps:

  • Merge your changes into the master branch
  • Choose a version for the update/release, following the guidelines of semantic versioning:
    Given a version number MAJOR.MINOR.PATCH, increment the:

    1. MAJOR version when you make incompatible API changes,
    2. MINOR version when you add functionality in a backwards-compatible manner, and
    3. PATCH version when you make backwards-compatible bug fixes.

    Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
    Note: If you want to release a stable version, it has to be at least 1.0.0, otherwise it will be considered unstable by composer and might lead to issues while installing it.

  • Update your composer.json with the new version (ie. “version”: “1.0.1”,) and commit the changes of the file.
  • Create a git tag with the new version:
    git tag 1.0.1
  • Push the changes and the tag:
    git push origin master
    git push --tag
  • That’s it! Now, you should be able to run composer update your-vendor/your-module and it will fetch your changes:
composer update lumbrales-software/magento2-first-module -v
Loading composer repositories with package information
Reading composer.json of lumbrales-software/magento2-first-module (1.0.2)
Importing tag 1.0.2 (1.0.2.0)
Reading composer.json of lumbrales-software/magento2-first-module (1.0.1)
Importing tag 1.0.1 (1.0.1.0)
Reading composer.json of lumbrales-software/magento2-first-module (master)
Importing branch master (dev-master)
Updating dependencies (including require-dev)
  - Removing lumbrales-software/magento2-first-module (1.0.1)
  - Installing lumbrales-software/magento2-first-module (1.0.2)
    Downloading: 100%         
    Extracting archive

Writing lock file
Generating autoload files

Note: Sometimes composer seems to keep cached the repository changes and won’t fetch your new code changes, I found a workaround for that by clearing the cache with the below command (run it as your own risk):

rm -fr ~/.composer/cache/*

That’s about it, you now should be able to release code changes nicely, and for every new code changes you just need to bump the version accordingly and remember to push both the version update in the composer.json and the git tag.

Good luck!

Let us know if you have any issues with it and we’ll try to assist you.

How-To Install Magento 2 on OpenShift – II

Magento 2

Magento 2

Last week we posted a guide to Install Magento 2 on OpenShift. It was just an experiment, and we didn’t intend to do anything else after we showed the proof of concept. However, since the guide has become very popular, we’ve decided to dig out a bit more into this and see if there was a less hacky way of install Magento 2 in OpenShift, and there  we are glad to say that we’ve managed to get it up and running with MySQL 5.7, by using another community cartridge.

It’s been very challenging due the hard disk limitation of 1GB, and we’ve had to delete everything that wasn’t absolutely necessary, as well as give up on performance in MySQL settings to squeeze a few more extra space, but eventually it worth the effort!

The steps to spot a new instance are almost the same as in our previous post, but this time we’ve simplified them even more within a simple bash script.

All you have to do is copy and paste this in a .sh file, set your API KEY & SECRET and run it from your favourite terminal (sh filename.sh). Remember that you need to have rhc installed and your Magento API Keys (check our previous post if you need any help with that):

#!/bin/bash

#CHANGE THIS SETTINGS
myapp='myapp' #App name
API_KEY='YOUR API KEY'
API_SECRET='YOUR API SECRET'
# Database settings, change them if you want
USERNAME='magento'
PASSWORD='123456'

regex='Git remote: \K(.*) (?=Cloned)'
echo Creating app...
newapp=$(rhc create-app $myapp http://cartreflect-claytondev.rhcloud.com/github/boekkooi/openshift-cartridge-nginx)
echo $newapp
GIT_REMOTE=`echo $newapp | grep -Po "$regex"`
echo $GIT_REMOTE

echo Setting env variables
rhc set-env API_KEY=$API_KEY API_SECRET=$API_SECRET OPENSHIFT_MYSQL_DB_USERNAME=$USERNAME OPENSHIFT_MYSQL_DB_PASSWORD=$PASSWORD -a $myapp

echo Installing cartridges
rhc cartridge add -a $myapp http://cartreflect-claytondev.rhcloud.com/github/icflorescu/openshift-cartridge-mysql
rhc cartridge add -a $myapp http://cartreflect-claytondev.rhcloud.com/github/boekkooi/openshift-cartridge-php

echo Cloning repository
git clone https://github.com/javilumbrales/magento2-openshift.git
cd magento2-openshift
git pull
git checkout magento-2

echo Deploying!
git remote add $myapp $GIT_REMOTE
git push $myapp magento-2:master -f

That’s it, your Magento 2 instance should be up and running on http://$myapp-$namespace.rhcloud.com.

The admin url and credentials are the same (/admin123 with admin / OpenShiftAdmin123)

Please, note that after deploying the app, the gear is almost out of space, so you would only be able to add minor data/changes to it. Hopefully, it should be enough for you to test it and see how it performs, and if you like it you could always upgrade your account to get some more space.

Enjoy!

Show me the code

How-To Install Magento 2 on OpenShift

MAgento 2If you are wondering how to try and install Magento 2 on a free hosting provider, keep reading. This is how we managed to deploy Magento 2 to Openshift without spending a penny!

This guide assumes you know OpenShift and have some basic experience with it, otherwise head to www.openshift.com, create an account and do some reading and basic tests.

Note that at the end of this post you’ll have the link to a repository that is almost ready to spin up a Magento 2 instance, but if you want to be aware of the caveats we’ll recommend you to keep on reading.

To begin with, we have the following “out of the box” issues:

  • Magento 2 requires PHP 5.5+ and OpenShift currently has official cartridges of up to PHP 5.4
  • Magento 2 requires shit lot of space, and OpenShift free gears are limited to 1GB
  • Magento 2 requires MySQL 5.6+ and OpenShift currently supports up to MySQL 5.5 official cartridges

To solve the PHP version issue, we firstly tried a community cartridge that installs Nginx and PHP 5.5/5.6. The cartridge itself worked flawlessly, but unfortunately then we run out space while running composer install after deploying the Magento 2 repository. We searched for something else and tried to run Magento 2 in PHP 5.4 (it was a bad idea btw). Eventually, we had to come back to that cartridge, as it seemed to most straightforward way to go. There doesn’t seem to be much options out there to install PHP 5.4+ in OpenShift without manually downloading the source code and compiling it and that would be beyond the scope of our purpose.

Moving on to the next issue, the out of space problems: While deploying Magento 2 to OpenShift, you almost run out of space, and yet you have to run `composer install` to fetch all the dependencies, and there are quite a few. While running `composer install –no-dev`, it would fail at half way, showing an error: Disk quota usage exceeded. We tried clearing composer cached files to free up some space but still wasn’t enough.

After a lot of back and forth, we managed to install all the dependencies without running out of space, but at the price of deleting the rollback deployments ($OPENSHIFT_DEPLOYMENTS_DIR). Note that this is not recommended at all, as you are actually loosing many of the benefits of using OpenShift, but if you ever need to make some extra space, you can run the following command in the OpenShift instance (We actually added that command to the `post_deploy` action_hooks script (file .openshift/action_hooks/post_deploy of your app’s repository):

rm -fr $OPENSHIFT_DEPLOYMENTS_DIR/*/repo/

Now, we had enough space available to install the dependencies, but it is required to have an API KEY and an API SECRET to be able to fetch them from the Magento Repository. You can get your own key pair for free by following this guide (basically create an account in Magento, and click on Generate Keys inside the Developers section after logging in). In order to keep the keys outside the source code, we opted to set them as environment variables (API_KEY and API_SECRET) while starting the new app. You’ll see how to do this further down.

Last, but not least, the issue with the MySQL version. After some research we ended up with the conclusion that it will not be easy at all to install MySQL on OpenShift, not to mention that we were already struggling with the disk space available. That’s why we decided that, for the shake of the experiment, we would try to make Magento 2 run over MySQL 5.5. After all, it wouldn’t be that different, would it? It turns out it is actually quite different. We were expecting to get away with this by just amending the check for the MySQL version in the Magento code, but that wasn’t enough. While running the installer, it would crash while creating the tables due some incompatible definitions. It took a bit of hacking that we are not proud of, which basically allowed us to install it by doing the following changes:

  • Removing the checks for the MySQL version
  • Amending a few table definitions that were not compatible with MySQL5.5:
    • The first error was: “There can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause”. In order to fix it we basically got rid of the CURRENT_TIMESTAMP that was being added to the updated_at columns.
    • The second error was: “The used table type doesn’t support FULLTEXT indexes”. To fix it we got rid of the FULLTEXT indexes on the InnoDB tables.

After the above changes, we managed to install Magento 2 successfully. However, as a result of this, the database and the app’s behaviour might (and will) not be stable, mostly in regards of search functionality, but also there will be a few datetime columns that will not be updated accordingly after data updates. Hopefully, all of this will not be needed soon as OpenShift provides an official MySQL 5.6 cartridge.

The “good news” are that our repository automatically provides the database schema that we managed to generate (which is compatible with mysql 5.5), so the source code of Magento 2 is not modified at all, as the changes were only required for the installation stage.

After this, we had a few permissions issues, and we had to add the config.php and env.php files (generated by the installation) to the repository with a default config and the database credentials taken from the environment variables. As Magento 2 doesn’t use local.xml anymore, it was fairly easy to replace the hardcoded database credentials on the env.php file by getenv(‘OPENSHIFT_MYSQL_DB_USERNAME’), etc. and it just worked. This is very convenient as well because you don’t have to make any changes on this files for it to work out of the box in your app, as it will get the database credentials from OpenShift provided environment variables.

Ok, enough talking. We’ve put all this together in a repository for you, so now you can almost almost plug & play it by following the below steps.

How to spot a new OpenShift app with Magento 2

In order to start a new app, all you need to have is:

Then, open a termial and run the below commands:

# Create a new OpenShift app with NGINX as web server
# Replace "123" by your actual api key & secrets and $myapp with your preferred app name.
# Write down the "Git Remote" ssh url that will be shown once the command finishes.
rhc create-app $myapp http://cartreflect-claytondev.rhcloud.com/github/boekkooi/openshift-cartridge-nginx API_KEY=123 API_SECRET=123
# Install PHP 5.6 cartridge into the app
rhc cartridge add -a $myapp http://cartreflect-claytondev.rhcloud.com/github/boekkooi/openshift-cartridge-php
# Install mysql-5.5 cartridge into the app
rhc cartridge add -a $myapp mysql-5.5
# Clone our repository and push it to your app, we will take care of everything for you 🙂
git clone https://github.com/javilumbrales/magento2-openshift
cd magento2-openshift
# Remember to replace $myapp by your app's name and YOUR_GIT_REMOTE by your actual repository url, the one that you got when you created the app (ie. should be something like ssh:[email protected]/~/git/yourappname.git/)
git remote add $myapp YOUR_GIT_REMOTE
git push $myapp master -f

This whole process will take a while, but you don’t have to do anything other than copy and paste the commands and wait. Hopefully, you’ll see the following success message at the end:

remote: Deployment completed with status: success

Done! You should now be able to browse your Magento 2 store on http://$myapp-$mynamespace.rhcloud.com/.

Probably the statics will be missing, and in order to fix that SSH into your app (`rhc ssh $myapp`) and run the below command to deploy the statics:

php $OPENSHIFT_REPO_DIR/public/bin/magento setup:static-content:deploy

Note that you might need to run it a few times, as, for some reason that we couldn’t figure out, sometimes it just crashes. Retrying a few times seems to do the trick though.

Eventually, you should see at the end of the execution the following message:

New version of deployed files: XXXXXXXX

And that’s it! Now your Magento 2 app should be completely up and running! The admin is available on http://$myapp-$mynamespace.rhcloud.com/admin123 with the credentials: admin / OpenShiftAdmin123.

Good luck!

Conclusions

It was a quite painful process, and it took several hacks to get Magento 2 running on OpenShift, but mostly due the outdated official cartridges for both PHP and MySQL. Hopefully, they’ll be updated soon and we should be able to make this process much easier and quicker.

After all, the app seems to perform very well in terms of speed, and we haven’t spend any time optimising any settings. So it certainly looks that OpenShift might potentially be a very interesting option to host a Magento 2 store. As per today, this is only on an experimental stage, to say the least, but it was a very challenging and interesting task to find our ways around the many limitations that we faced, and we finally did it! (well, kind of :)).

Troubleshooting

If you get this error while deploying the app:

remote: [Composer\Downloader\TransportException]
remote: The ‘https://repo.magento.com/archives/magento/composer/magento-composer-1.0.2.0.zip’ URL required authentication.
remote: You must be using the interactive console to authenticate

Make sure that your API_KEY and API_SECRET env variables are set with the proper values. You can re-set them with the following command:

rhc set-env API_KEY=ENTER_YOUR_API_KEY_HERE API_SECRET=ENTER_YOUR_API_SECRET_HERE -a $myapp

Demo

Finally, if you just want to have a look at the resulst, you can find here our Magento 2 app running on OpenShift: http://demo-mage2.rhcloud.com/

You can also login to the admin on http://demo-mage2.rhcloud.com/index.php/admin_1ep725/ (User: demo Pass: demo123)

Show me the code!

How-to create a Magento 2 Composer Module

MAgento 2If you are a developer new to Magento 2, you are probably wondering how to create a Magento 2 Composer Module. Hopefully the below steps will be useful to you. This is how I managed to do it, eventually.

First of all, you should familiarise yourself with composer, and it’s basic usage.

This guide asumes you’ve already installed and configured a vanilla Magento 2 project. If you haven’t done that yet, I’d advise you to check this out and follow the official guide. Feel free to drop me a line if you have any issues.

I’ve found quite a few sites that show how to create a Magento 2 module “the old way”, by copying the module contents inside app/code folder (note that in Magento 1.x it used to be app/code/codePool, but that doesn’t exist anymore in Magento 2).

However, as I’ve recently been working on Symfony 2 projects, I’m quite used to composer, and it’s usage to easily integrate third party components/packages. So, once I saw the file “composer.json” on the root directory of Magento 2, I knew that it should be the way forward in order to develop Magento 2 extensions.

Let’s get started. I’m assuming you’ve developed already your Magento 2 extension, as the purpose of this guide is to show how to package the extension within a composer (sub) module, and the creation of an extension would go beyond the scope of this post. Check this guide to find out how to create a Magento 2 extension, or just clone it’s source code from github.

  • To begin with, you should create a new (github/bitbucket…) repository, and place your source code inside.
  • In the root folder, create a new file named composer.json. This fille will describe the details of your package. This is how mine looks like:
    {
        "name": "lumbrales-software/magento2-first-module",
        "description": "Hello world Magento 2 Composer Module!",
        "require": {
            "magento/project-community-edition": "*"
        },
        "type": "magento2-module",
        "version": "0.1.0",
        "extra": {
            "map": [
                [
                    "*",
                    "Yourcompany/YourModule/"
                ]
            ]
        },
        "authors": [
            {
                "name": "Javi Lumbrales",
                "homepage": "https://www.lumbrales-software.com/",
                "role": "Developer"
            }
        ]
    }

    The most important parts described below:

    • The name of the package: This will serve as identifier of your package, and will be used later on.
    • Requirements: We’ve added “magento/project-community-edition” as a requirement, as this module requires Magento 2 in order to work.
    • Type: We specify “magento2-module” so that it’s contents are copied to app/local once the package is installed (more about this later).
    • Version: Not important, as we’ll use git tagging to handle the composer module updates.
    • Extra: – Map: This tells composer to copy all (*) contents into app/code/LumbralesSoftware/HelloWorld, so the root of your extension should be structured already. This means that you don’t need to keep your files under app/code/YourCompany/YourModule in your repository. Instead, they should be in the root straightly. This is how it your repository should look like:

Repository Root:
Block/
Controller/
etc/

  • Push all your code to the master branch (I’ll explain the tagging on a separated post later on)
  • Go to the root folder of your Magento 2 installation, and type the following commands in your terminal:
composer require magento/magento-composer-installer

This is a submodule required to properly install third party Magento 2 extensions as composer submodules (Remember the type:magento2-module mentioned at the beginning).

  • Then, as your module is still not in a packages repository such as packagist.org, or packages.magento.com, you need to specify your own VCS repository so that composer can find it. In the repositories section of the composer.json file of the Magento 2 project (if the repositories section doesn’t exist, create it, else, add your repository at the end) add the following:
 "repositories": [
    {
      "type": "vcs",
      "url": "https://github.com/youraccount/yourmodule-repository"
    }
  ],
  • Since your package is in a development stage, you will need to add the minimum-stability as well to the composer.json file:
"minimum-stability": "dev",

Note that this could be removed later on, once we have properly tagged/released a stable version of our package/extension.

  • After that, you should be able to install your module as follows (remember how you named it):
composer require your-package/name #In our example the name was lumbrales-software/magento2-first-module

The above command should download your module and copy it’s content to app/code/YourCompany/YourModule

  • Last, but not least, you will need to enable your module and clear Magento cache. Add a new entry in app/etc/config.php, under the ‘modules’ section (Magento 2 project):
    'Yourcompany_YourModule' => 1,

Note that this name should match with the name that you’ve put on the composer.json, section extra -> map.

That’s it, after that your module should be ready to go.

You can download the source code of my Magento 2 Composer Module sample here.

I hope it helps, and let me know if you have any issues!

In my next post I’ll show a basic usage of git tagging and composer to be able to easily release your module updates.

Credits:
https://www.ashsmith.io/2014/12/simple-magento2-controller-module/
https://alankent.wordpress.com/2014/08/03/creating-a-magento-2-composer-module/