DeadTrees 1.1: More Cover Sources

An update to DeadTrees, my WordPress plugin for sharing the books I read, is now live in the WordPress Plugin Repository. This update does one major thing: diversifies the sources of cover images.

What happened was Amazon introduced a quota on the API that the original DeadTrees was using to fetch cover art. The API quota is based on how many sales a user has, and I’m famous enough to have many sales, so my API access was eventually cut off. When I started posting my backlog of books I really wanted cover art, so I polished off the (very) dusty code and got to work.

Version 1.1 of DeadTrees maintains support for Amazon cover art and adds support for fetching cover art from OpenLibrary.org and LibraryThing. There’s a setting to try Amazon first or last, and the plugin tries to be smart about when to try OpenLibrary.org or LibraryThing, (it prefers OpenLibrary.org, as they seem to provide larger images, and don’t require an API key).

If you want to see DeadTrees in action take a look at the list of books I have read, which is powered by DeadTrees. For support post in the WordPress.org forum, and for bugs & feature requests post in the same forum or create a Github issue.

Happy Reading!

Remove a model’s Global Scopes in Laravel Nova

I am working on a Laravel project that has moderated reviews. Most of the time we only deal with reviews that have been approved, so the Review model has an ApprovedScope Global Scope to only show approved review, but staff members need to see un-approved reviews in our Nova admin so that we can approve, (or reject), reviews. Removing a Global Scope from a model for all of Nova is trickier than it appears at first glance, but there’s at least one way to do it, and maybe more.

Things that do not work

Call withoutGlobalScopes() in the Nova::serving() callback

Googling “remove global scope nova” turns up a post on Medium called Add or remove global scopes in Nova. Sounds promising! It claims we can remove global scopes by adding this to the boot method of the `NovaServiceProvider:

class NovaServiceProvider extends NovaApplicationServiceProvider
{
    /**
     * Bootstrap any application services.
     *
     * @return void
     */
    public function boot()
    {
        Nova::serving(function () {
            \App\Rating::withoutGlobalScopes();
        });
    }
}

Cool! But it doesn’t work. When adding a global scope to a model using the model’s boot method like the Laravel documentation shows, the model’s boot method is called after the Nova::serving() callback. So in this situation we remove nothing from the Rating, then add a global scope later when the model is booted.

Call withoutGlobalScopes() in the model’s booted event

If a model has a global scope added during its boot then we should be able to remove it during the ‘booted’ event – or at least that’s what I thought, so I tried this in NovaServiceProvider::boot :

Nova::serving(function() {
    Event::listen('eloquent.booted: App\Rating', function($rating) {
        Rating::withoutGlobalScopes();
    });
}); 

No dice. I’m not certain why this doesn’t work, but some XDebug spelunking shows me that:

  1. There’s no withoutGlobalScopes() method on Laravel models, but when the model gets a method call that it doesn’t recognize it passes it to the model’s Builder instance.
  2. The Builder does have a withoutGlobalScopes method, so we don’t get an InvalidMethodException. In the Builder’s withoutGlobalScopes call the ApprovedScope is removed from the Builder’s list of global scopes.
  3. But the Global Scope is still applied to the query. My best guess is that a different query builder is used to actually generate results, or that later, when the models are about to be retrieved from the DB, the model re-passes the list of Global Scopes to the builder – and since we haven’t removed the Global Scope from the model it gets re-added to the Query Builder. If anyone knows what’s actually going on here I would love to know in the comments or on Twitter.

Something that does work

We need to remove the Global Scope after the model has booted and prevent it from being passed or re-passed to the Query Builder. How about a Model::withoutGlobalScopes method? The model’s addGlobalScope method comes from the Illuminate\Database\Eloquent\Concerns\HasGlobalScopes trait, and stores the global scopes in a static::$globalScopes[static::class] array. Creating our own HasRemovableGlobalScopes trait, with withoutGlobalScope and withoutGlobalScopes methods that mirror the signature of the Illuminate\Database\Eloquent\Builder withoutGlobalScope and withoutGlobalScopes methods can solve the problem, (also available as a Gist):

<?php

namespace App\Concerns;

use Closure;
use Illuminate\Database\Eloquent\Scope;
use Illuminate\Support\Arr;

trait HasRemovableGlobalScopes {

	/**
	 * @param  \Illuminate\Database\Eloquent\Scope|string  $scope
	 */
	public static function withoutGlobalScope( $scope )
	{
		if (is_string($scope) && is_array(static::$globalScopes[static::class])) {
			Arr::forget(static::$globalScopes[static::class], $scope);
		} elseif ($scope instanceof Closure) {
			Arr::forget(static::$globalScopes[static::class], spl_object_hash($scope));
		} elseif ($scope instanceof Scope) {
			Arr::forget(static::$globalScopes[static::class], get_class($scope));
		}
	}

	/**
	 * @param \Illuminate\Database\Eloquent\Scope[]|string[] $scopes
	 */
	public static function withoutGlobalScopes( array $scopes = [])
	{
		if(empty($scopes)) {
			static::$globalScopes = [];
		} else {
			foreach($scopes as $scope) {
				static::withoutGlobalScope($scope);
			}
		}
	}
}

The withoutGlobalScope method mirrors the HasGlobalScopes::addGlobalScope method to remove a single global scope, and the withoutGlobalScopes method can accept an array of global scopes to remove or be called with no parameters to remove all global scopes, (the same as Builder::withoutGlobalScopes).

A drawback?

The fact that we have to do this to remove global scopes for Nova seems to be an oversight. I’m hopeful that withoutGlobalScope/withoutGlobalScopes methods will be added to future versions of Laravel. If that happens there will be a method name collision between the HasRemovableGlobalScopes trait’s methods and the first-party ones, so read the release notes if you’re going to use this method.

Something else that might work

After writing the HasRemovableGlobalScopes trait I realized it should be possible to create an additional global scope that undoes my original global scope. In this case something like this might work:

Nova::serving(function() {
    Event::listen('eloquent.booted: App\Rating', function($rating) {
        Rating::addGlobalScope(new AllRatingsScope);
    });
}); 

Where AllRatingsScope says to include all ratings. I haven’t tried this so don’t know what happens when there are two global scopes that specify opposite conditions. You might get no results, or the scope that’s applied last might win. Assuming that the scope applied last wins it’s still important to use the eloquent.booted event to make sure the “undo” scope is added after the original scope.

All together now

I’m sticking with the HasRemovableGlobalScopes trait. I feel like actually removing the Global Scope is more logical than adding another scope to undo what the first one does. Using the HasRemovableGlobalScopes trait, this is what my whole system looks like:

Apply the original global scope to the Rating, (or wherever you need a global scope), and use the HasRemovableGlobalScopes trait:

class Rating extends Model
{
	use HasRemovableGlobalScopes;

	// ... 

	protected static function boot()
	{
		parent::boot();
		static::addGlobalScope(new ApprovedScope);
	}

	// ...
}

Set the event listener in the NovaServiceProvider’s boot method to remove the global scope for Nova:

class NovaServiceProvider extends NovaApplicationServiceProvider
{
    /**
     * Bootstrap any application services.
     *
     * @return void
     */
    public function boot()
    {
	    Nova::serving(function() {

		    Event::listen('eloquent.booted: App\Rating', function($rating) {
			    Log::info('rating booted ' . request()->url());
		    	Rating::withoutGlobalScopes();
	        });
	    });

	    parent::boot();
    }
}

It works well for me. A possible enhancement would be to have my own App\Model class that extends Laravel’s Model and uses HasRemovableGlobalScopes, then extend my other models from that. At the moment I don’t have my own App\Model, and don’t have a lot of places where I need to share between all my models, but if I find I’m sprinkling too many use HasRemovableGlobalScopes lines around my code base I’ll make the change.

FormRequest object Validation for Laravel Nova Resources

Laravel Nova gives us developers a relatively easy way to set up an administration panel for a website or app, but it’s been publicly available for less than a year and is only at version 1.3.1, so there are a few rough edges, like the inability to use FormRequest objects for validation when updating models, (called “Resources” in Nova parlance).

FormRequest objects are a way to detach form validation from specific controllers or routes so that the validation logic can be re-used, for example by using the same FormRequest object to validate updates made on a website or through an API. But they’re not supported by Nova so validation code needs to be re-written for Nova resources, or does it?

Comparing Laravel to Nova Validation

Validation rules in Nova are the same as the rest of Laravel, with the at least one tiny change in syntax, but instead of being passed around as one big list, each field has validation attached directly to it when defining the field. Here is the difference between rules in a FormRequest, (it’s the same whether using a FormRequest, Validator object, or in a controller), and rules for Nova fields:

Rules using a FormRequest

namespace App\Http\Requests;

use Illuminate\Foundation\Http\FormRequest;
use Illuminate\Validation\Rule;

class StorePostRequest extends FormRequest
{
     // ....
	
     /**
     * Get the validation rules that apply to the request.
     *
     * @return array
     */
    public function rules()
    {
        // note all rules are returned as an array
    	return [
	        'title' => 'required|unique:posts|max:255',
	        'body' => 'required',
	    ];
    }

    // ....
}

Rules using Laravel Nova

namespace App\Nova;

use Laravel\Nova\Fields\ID;
use Illuminate\Http\Request;
use Laravel\Nova\Fields\Text;
use Laravel\Nova\Fields\Textarea;


class Post extends Resource
{

    // ...
	
    /**
     * Get the fields displayed by the resource.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function fields(Request $request)
    {

    	return [

    		// ...
    		
                // In Nova the rules are attached to fields, which themselves are an array.
    		Text::make('Title')->rules('required', 'unique:posts', 'max:255'),
    		Textarea::make('Body')->rules('required')

    		// ...

    	];
    }

    // ...
}

The “Laravel Way” to use a FormRequest for validation would be to type hint the $request passed to the fields method like this, (but it doesn’t work, apparently ):

namespace App\Nova;

use Illuminate\Http\Request;
use App\Http\Requests\StorePostRequest;


class Post extends Resource
{
    // ...

    // This type-hint produces a fatal error
    public function fields(StorePostRequest $request)
    {
    	return [
    		// ...
    	];
    }

    // ...
}

I’m actually not sure why the error is produced. The error I get says that Post@fields expects an instance of Illuminate\Http\Request, and that StorePostRequest isn’t one, but I am pretty sure it is an instance of Illuminate\Http\Request. Either way it doesn’t work, and that’s the expected behaviour, so a workaround is needed.

A Workaround

One way to only write validation rules once is to move them into a static method on the FormRequest object that has an optional $rule parameter to fetch one rule at a time if needed. Since Nova allows separate creationRules and updateRules for each field there should be three corresponding static methods on the FormRequest, and the non-static rules method can merge the rules for the rest of Laravel if needed.

Here’s the FormRequest class I’m using to provide validation rules for adding and editing users in an application I’m working on:


namespace App\Http\Requests;

use Illuminate\Foundation\Http\FormRequest;
use Illuminate\Validation\Rule;
use App\Rules\Postalcode;


class StoreUserRequest extends FormRequest
{
    /**
     * Determine if the user is authorized to make this request.
     *
     * @return bool
     */
    public function authorize()
    {
	    // Users authorized to make the request are:
	    // - users updating themselves.
	    // - staff
	    // - guests creating a new user.


        return true;
    }

    /**
     * Get the validation rules that apply to the request.
     *
     * @return array
     */
    public function rules()
    {

    	$rules = self::ruleGetter($this);


	    if(empty($this->user())) {
		    $rules = array_merge_recursive($rules, self::creationRuleGetter($this));
	    } else {
		    $rules = array_merge_recursive($rules, self::updateRuleGetter($this));
	    }

	    return $rules;
    }

    public static function ruleGetter( $request, $rule = null ) {

	    $rules = [
		    'email' => [
			    'required',
			    'string',
			    'email',
			    'max:255'
		    ],
		    'nickname' => [
			    'required',
			    'max:255'
		    ],
		    'job_title' => [
		    	'sometimes',
		        'max:255'
		    ],
		    'organization' => [
			    'sometimes',
			    'max:255'

		    ],
		    'state' => [
		    	'sometimes',
			    'nullable',
			    'in:AL,IS,TO,FS,TA,TE,S'
			],
		    'postalcode' => [
			    'sometimes',
			    new Postalcode
		    ],

		    'phone' => [
		    	'max:255'
		    ]
	    ];

	    if(!empty($rule)) {
	    	if(isset($rules[$rule])) {
			    return $rules[$rule];
		    }
	    	return '';
	    }
	    return $rules;
    }

    public static function creationRuleGetter( $request, $rule = null ) {
	    $rules = [
	    	'email' => [
	    		'unique:users,email'
		    ],

	        'nickname' => [
	        	'required',
		        'unique:users,nickname'
	        ],

		    'password' => [
			    'sometimes',
			    'required',
			    'string',
			    'min:8',
			    'confirmed'
		    ],
	    ];


	    if(!empty($rule)) {
		    if(isset($rules[$rule])) {
			    return $rules[$rule];
		    }
		    return '';
	    }
	    return $rules;

    }

    public static function updateRuleGetter( $request, $rule = null ) {


    	$rules = [
    		'email' =>  [
		        Rule::unique('users', 'email')->ignore($request->user()->id)
		    ],

		    'nickname' => [
			    Rule::unique('users', 'nickname')->ignore($request->user()->id)
		    ],

		    'password' => [
			    'sometimes',
			    'nullable',
			    'string',
			    'min:8',
			    'confirmed'
		    ],
	    ];

	    if(!empty($rule)) {
		    if(isset($rules[$rule])) {
			    return $rules[$rule];
		    }
		    return '';
	    }
	    return $rules;
    }

}

Then, in the Nova resource the rules can be attached to individual fields like this:

<?php

namespace App\Nova;

use Laravel\Nova\Fields\ID;
use Illuminate\Http\Request;
use Laravel\Nova\Fields\Text;
use Laravel\Nova\Fields\Boolean;
use Laravel\Nova\Fields\Gravatar;
use Laravel\Nova\Fields\Select;
use Laravel\Nova\Fields\Password;
use Laravel\Nova\Panel;
use Laravel\Nova\Fields\DateTime;
use Laravel\Nova\Fields\Textarea;
use Laravel\Nova\Fields\Place;
use Laravel\Nova\Http\Requests\NovaRequest;


use App\Http\Requests\StoreUserRequest;

class User extends Resource
{

    /**
     * The model the resource corresponds to.
     *
     * @var string
     */
    public static $model = 'App\\User';


    // ....


   /**
     * Get the fields displayed by the resource.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function fields(Request $request)
    {

        return [
            ID::make()->sortable(),

            Gravatar::make(),

	        Text::make('Email')
		        ->sortable()
			    ->withMeta( [
			    	'extraAttributes' => [
			    		'type' => 'email'
				    ]
			    ])
		        ->rules(StoreUserRequest::ruleGetter($request, 'email'))
		        ->creationRules(StoreUserRequest::creationRuleGetter($request, 'email'))
		        ->updateRules('unique:users,email,{{resourceId}}'),

		    Password::make('Password')
		            ->onlyOnForms()
		            ->rules(StoreUserRequest::ruleGetter($request, 'password')),

		    Text::make('Name', 'name')
		        ->sortable()
		        ->rules(StoreUserRequest::ruleGetter($request, 'name')),

		    Text::make('Preferred Name')
		        ->rules(StoreUserRequest::ruleGetter($request, 'preferred_name'))
		        ->hideFromIndex(),

		    Text::make('Nickname')
		        ->rules(StoreUserRequest::ruleGetter($request, 'nickname'))
		        ->creationRules(StoreUserRequest::creationRuleGetter($request, 'nickname'))
		        ->updateRules('unique:users,nickname,{{resourceId}}')
		        ->hideFromIndex(),

		    Text::make('Job Title')
		        ->rules(StoreUserRequest::ruleGetter($request, 'job_title'))
		        ->hideFromIndex(),

		    Text::make('Company Name', 'organization')
		        ->rules(StoreUserRequest::ruleGetter($request, 'organization'))
		        ->hideFromIndex(),

		    Place::make('Address')
		        ->rules(StoreUserRequest::ruleGetter($request, 'address'))
			    ->countries(['US', 'CA'])
			    ->secondAddressLine('address2')
			    ->city('city')
			    ->state(['stateprov'])
			    ->postalCode('postalcode')
		        ->hideFromIndex(),

		    Text::make('Address Line 2', 'address2')
		        ->rules(StoreUserRequest::ruleGetter($request, 'address2'))
			    ->hideFromIndex(),

		    Text::make('City')
		        ->rules(StoreUserRequest::ruleGetter($request, 'city'))
		        ->hideFromIndex(),

		    Select::make('State or Province', 'stateprov')
			    ->options(get_stateprovs())
			    ->rules(StoreUserRequest::ruleGetter($request, 'stateprov'))
			    ->hideFromIndex(),

		    Text::make('ZIP or Postal Code', 'postalcode')
		        ->rules(StoreUserRequest::ruleGetter($request, 'postalcode'))
			    ->hideFromIndex(),

		    Text::make('Phone')
		        ->rules(StoreUserRequest::ruleGetter($request, 'phone'))
			    ->withMeta(['extraAttributes' => [
			    	'type' => 'tel'
			    ]])
		        ->hideFromIndex(),

        ];
    }

    // ....

}

Instead of writing out actual strings of rules for the Nova resource, the StoreUserRequest::ruleGetter method supplies the rule, allowing one canonical repository of rules for use with Users.

An Exception

There is an exception to the beauty: Enforcing unique values in a table, but ignoring a certain user ID, (for example when updating a user profile the user should not see a “This E-mail is already in use” error if the E-mail isn’t changed). Nova handles it like this:

// Simple and elegant 
Text::make('Email')
->rules('unique:users,email,{{resourceId}}')

Non-Nova Laravel handles it like this:

Rule::unique('users')->ignore($user->id, 'user_id')

But when declaring Nova fields a $user object, or the ID of the model that is being updated, is not available so this does not work. I have worked around the problem by writing out these few cases in the updateRules() method of the Nova field, as in this excerpt from above:

// Excerpt from full Nova Resource definition.
// Most Rules use one of the ruleGetter methods on StoreUserRequest
// but updateRules are just typed out to take advantage of Nova syntax.
Text::make('Email')
->rules(StoreUserRequest::ruleGetter($request, 'email'))
->creationRules(StoreUserRequest::creationRuleGetter($request, 'email'))
->updateRules('unique:users,email,{{resourceId}}'),

Other Possibilities

Ideally it would be possible to have the FormRequest object do the validation itself, without having to pass individual rules to Nova fields. Another possibility is to turn the Request that’s passed to a Resource’s fields method into a FormRequest, or to instantiate a completely new FormRequest object, and let the FormRequest object handle validation. Unfortunately I don’t know where we would do this, (the fields method doesn’t seem right), and I don’t know if the redirects and error messages generated by FormRequest validation would work properly with Nova. I would love to hear if someone can make this work.

Conclusion

I sincerely hope that Nova will soon allow automatic validation with FormRequest objects, but until that happens it is possible to keep code almost DRY and feed individual rules to Nova fields with ruleGetter methods on our FormRequests.

WP e-Commerce to WooCommerce

There’s a pair of websites that I have looked after for a long time. They belong to a company that sells a few products, and have had quite a few sales over the years. This fall we’re doing some major renovations, and one of the things being changed is the underlying E-Commerce plugin for WordPress; we are changing from WP e-Commerce to WooCommerce. I expected that after installing WooCommerce it would offer to convert my WPeC store into a WooCommerce store. I was wrong. There was an official converter plugin but it was abandoned long ago.

Options to change a site from WP e-Commerce to WooCommerce

It looks like there are three ways to change a site from WP e-Commerce to WooCommerce:

  1. Use a commercial service like Cart2Cart. Cart2Cart looks great, but is made to move a store from one site to another, and I need an in-place conversion. They also charge per-migration, so if I want to migrate my localhost, modify the theme for WooCommerce, then migrate the live site, I’ll have to pay 1.5 times, (they give a 50% discount for re-migrating a site). With two sites my bill would have been around US$450, high enough for me to look at other, more complicated options.
  2. Create or update something like the old WooCommerce migration plugin.
  3. Manual Migration: This would involve a lot of SQL.

Cart2Cart looks ill-suited for my use-case, and expensive. Manual migration looks difficult, highly prone to errors, and not easily repeatable. This leaves creating or updating a migration utility. This migration script by Carl Hughes is bit newer than the WooCommerce one, promises to do more out of the box, and quick read-through doesn’t show anything too mysterious. It was my starting point.

Updating and Testing

It took three times as long as expected to update & test the migration plugin. Here’s what I learned, for anyone else that might be considering the same type of process:

  • This should go without saying, but this migration is destructive. If it fails the database will be in a strange state between WPeC and WooCommerce. Have a backup.
  • Since migrating from WPeC to WooCommerce changes the database, but only the database, (no media files are touched), a backup of the WordPress database is required. A lot of time will be spent reloading from the backup, so put it somewhere easy to get. Mine was on my desktop. Bonus points: a short bash script to reload the DB from the backup will save development time.
  • Make the backup be the exact state that the migration should run from. It is not efficient to disable a plugin in the WP Admin every time the database is reloaded.
  • Some plugins really slow down the migration. The most obvious ones are WP e-Commerce itself, and any Varnish caching plugin. If they are disabled the migration will run faster.
  • Each payment gateway, (plugin for a payment processor, in WPeC terminology), stores transaction data a little differently. I added the ability to port data from Authorize.Net credit card transactions, but separate routines need to be written for other processors.

Updated Migration Plugin

My fork of the migration plugin is up on Github, and a I sent a pull request to the original author so he can include my work if he wants. If you need to migrate from WP e-Commerce to WooCommerce hopefully this can help a bit. Read the readme. Migrating is harder than it should be, but it is possible.

Changing the Name and Path of the Active WordPress Theme Without Breaking Theme Settings

After changing a brand name, but little else, on a WordPress site we can end up with a WordPress theme called “oldbrand” on a site called “New Brand.” This leads to weird things like the brand’s logo being at the URL newbrand.com/wp-content/themes/oldbrand/img/newbrand-logo.png. Oops. SEOs must cry a little when they see something like that. This is a situation one of my clients ended up in after a name change, but it ends now.

We can’t simply change the theme directory from themes/oldbrand to themes/newbrand and update the active theme in the Appearance section of the wp-admin because theme-related settings are tied to the theme directory in the database. This includes things like widget placement; a different theme probably has very different widget areas so widget placement should be stored per-theme. The same goes for any theme settings & customizations. To get around this we need to update the database so WordPress thinks that the new theme name is the theme name we’ve always been using.

A word of caution: back up your database before trying this. We’re going to run raw queries. If something breaks you may end up in an in-between limbo, and restoring from backup is the quickest way out.

After diving through the database of a few sites this morning these queries make it possible to rename the theme directory, and optionally rename the theme in style.css:

# Update the main theme options
UPDATE wp_options SET option_value='new-theme-directory' WHERE option_name='template';
UPDATE wp_options SET option_value='new-theme-directory' WHERE option_name='stylesheet';
UPDATE wp_options SET option_name='theme_mods_new-theme-directory' WHERE option_name='theme_mods_old-theme-directory';

# If also updating the Theme Name in the theme's style.css
UPDATE wp_options SET option_value='New Theme Name' WHERE option_name='current_theme';  

# If any posts reference assets in the theme such as images or logos.
UPDATE wp_posts SET post_content=REPLACE(post_content, 'themes/old-theme-directory', 'themes/new-theme-directory' );
UPDATE wp_postmeta SET meta_value=REPLACE(meta_value, 'themes/old-theme-directory', 'themes/new-theme-directory');

# If WordFence is used
UPDATE wp_wfConfig SET val=REPLACE(val, 'themes/old-theme-directory', 'themes/new-theme-directory');

# Delete a couple of transients that store references to the old theme directory.
DELETE FROM wp_options WHERE option_value='_site_transient_theme_roots';
DELETE FROM wp_options WHERE option_value='_site_transient_update_themes';

Protips:

  • Try to run these queries at nearly the same time as the theme directory is renamed. Bonus points for making a script that does it all nearly instantly.
  • If an installation stores transients somewhere other than the DB, then clearing them in the DB won’t work, (clearing the transients may not be strictly required).
  • The sample code uses the default wp_ table prefix. When working on a site with a different prefix, use that.

Hopefully this helps someone finish their Googling session and get on with a rename!