We are currently in the process of building a new website in Plone 5. The current site is running on Plone 4.3, but was originally built in Plone 2, and upgraded various times along the way. Instead of migrating the site again, we've decided to start fresh with a new buildout, new theme, and mostly new content. While much of the site content will be created new, we still want to maintain existing news items and blog posts, and turn them into Dexterity items. Hence the need for a Transmogrifier migration...
In the Calvin and Hobbes universe, the Transmogrifier is an invention of Calvin's that would turn one thing into another. Similarly, collective.transmogrifier provides support for building pipelines that turn one thing into another, hence greatly facilitating content import and export. This is a life saver when you need to migrate content from one site to another, regardless of the CMS platform.
Exporting content from the old site turned out to be quite easy with collective.jsonify. Full instructions and options are clearly detailed on that site, but the basic process for exporting all content is to:
Install collective.jsonify into the buildout of the old site
Add an External Method at the root of the site with the following properties:
export_contentcollective.jsonify.json_methodsexport_contentGo to http://[your site]/export_content
See the instance log output for where the export throws the content (mine went into /tmp)
Copy the numbered folders from the export into the new buildout, in a folder at the root called content-import and add this to your .gitignore.
Now you are ready to start the process of importing the content into the site. This part requires writing a pipeline that determines how to handle the content coming in to the new site. For this we recommend creating a custom package just for the migration code. This makes it easy to remove it later after everything is migrated.
First install the necessary add-ons into the buildout. You will need collective.jsonmigrator and transmogrify.dexterity. You may also need to pin zope.app.container < 4.0 because 4.0 requires a lot of newer Zope packages than what are pinned for Plone 5.1.
Use mr.bob to create a custom package. In a virtualenv:
$ pip install mr.bob $ pip install bobtemplates.plone $ mrbob -O yourproject.migration bobtemplates:plone_addon
This will ask a series of questions. Create a 'Basic' add-on, and supply the author information.
Below is part of our import_content.cfg. Each section specifies a blueprint section to be used and the information of what you want to manipulate.
[transmogrifier] # Each pipeline corresponds to a blueprint section below pipeline = jsonsource limittypes removeid pathfixer movenews constructor copyuid deserializer schemaupdater workflowhistory datesupdate properties logger [jsonsource] # Specify the source folder in the buildout we are importing from blueprint = collective.jsonmigrator.jsonsource path = content-import [limittypes] # Only import News Items, Blog Posts, Images, and Files # Everything else will be skipped if it doesn't match blueprint = collective.transmogrifier.sections.condition condition = python:item['_type'] in ['News Item', 'BlogPost', 'Image', 'File'] [removeid] # The jsonify will have the id we want to use in the `_id` field # so remove the one from the data we don't want to use blueprint = collective.transmogrifier.sections.manipulator delete = id [pathfixer] # Remove the old Plone Site ID from the paths so we can import into any new Plone # site reguardless of its site ID. blueprint = plone.app.transmogrifier.pathfixer stripstring = /SFUPSite [movenews] # Relocate the path to the main news folder, old site path vs. new site path blueprint = collective.transmogrifier.sections.inserter key = string:_path value = python:item['_path'].replace('/company/news/news', '/company/whats-happening/company-news') [constructor] # Here's where the actual magic happens, create the content object in the site blueprint = collective.transmogrifier.sections.constructor [copyuid] # Needed so that the schemaupdater can set the UUID correctly blueprint = collective.transmogrifier.sections.manipulator keys = _uid destination = string:plone.uuid [deserializer] # If the data was contained inside of an attached JSON file, stuff that data # back into the pipline for the next step. blueprint = transmogrify.dexterity.deserializer [schemaupdater] # Now updated the created item from the data in the dictionary that has been # passed down the pipeline blueprint = transmogrify.dexterity.schemaupdater [workflowhistory] # Custom blueprints can also be created and used # For example, if you need to include workflow history blueprint = sixfeetup-plone5.migration.workflowhistory [properties] # Copy each item's zope properties into the the property sheet of the object blueprint = collective.jsonmigrator.properties [datesupdate] # Plone itself includes some nice blueprint sections that are ready to use # Set creation, modification and effective dates on the object blueprint = plone.app.transmogrifier.datesupdater [logger] # Critical to developing your piplines is the ability to see log output of what has happened # Use the logger blueprint section to pick and choose what to output into your logs blueprint = collective.transmogrifier.sections.logger name = IMPORTER level = DEBUG delete = text _datafield_file _datafield_image
This pipline is registered with the following ZCML:
<configure xmlns="http://namespaces.zope.org/zope" xmlns:transmogrifier="http://namespaces.plone.org/transmogrifier" xmlns:genericsetup="http://namespaces.zope.org/genericsetup" i18n_domain="collective.transmogrifier"> <include package="collective.transmogrifier"/> <include package="collective.transmogrifier" file="meta.zcml"/> <transmogrifier:registerConfig name="sixfeetup-plone5.migration.import_content" title="Import sixfeetup-plone5 content" description="This pipeline imports content to single Plone site" configuration="import_content.cfg"/> </configure>
One more step needs to be done to hook up the pipeline to the default profile. Add a file into the profiles/default folder called transmogrifier.txt. Inside the file, only put the name of the pipeline. From the example above, this is sixfeetup-plone5.migration.import_content.
To run the import, go to portal_setup in the ZMI to run the migration package's default profile Make sure to test the import and adjust for what you need before running in production.
Six Feet Up, Inc.
11208 Windermere Blvd.
Fishers, IN 46037 - USA
Phone: +1 (317) 861-5948
Email: info@sixfeetup.com
