Workflow

Describes the recommended workflow in steps in order to build a feature with the most efficient use of time, showcasing how Lucid enhances productivity and efficiency.

Building applications is as similar in concept as building any artefact made of multiple pieces such as houses, towers, castles etc. which are all built on the basis of the same construct - stone walls. For that we need material, structure, measurements and most importantly sequence; for we need the base layer of stones and cement to build upon, and it is built row by row.

In this guide we show the recommended process of building robust (yet flexible) blocks that make our application whole, ensuring ideal levels of productivity and efficiency by outlining the composition and interaction of our components prior to indulging into implementation - Thinking time as described in Peopleware is the objective of this workflow.

Writing Features

  1. Generate a Feature class, which will create its test class along.

  2. Open the Feature class and describe the steps to achieve the feature's functionality in TODO items such as:

 class UpdateResourceFeature extends Feature
 {
     public function handle(Request $request)
     {
         // 1. @TODO: validate request input (format & length)

         // 2. @TODO: verify access to resource

         // 3. @TODO: update resource

         // 4. @TODO: save audit activity (async)

         // 5. @TODO: notify followers (async)

         // 6. @TODO: respond with update status
     }
 }

This approach gives the benefit of an organised and defined workflow, it enhances productivity, focus and visibility by summarising the steps as an overview of what it is to be implemented. In addition to the benefit of editors listing those when needed:

  1. Start by writing the test for the feature: it is a request to call the route with one of the acceptance cases with input params (if applicable) to execute this feature. At this step, we will have to create the route, controller and controller method to serve the feature, and add the serve(UpdateResourceFeature::class) line to the controller method.

  2. Now that we've written our first test case to execute the feature, it sure will fail because our Feature class is still empty. Now, we can start filling this feature with jobs by replacing each of these steps with the Job or Operation class that fulfils it, following the workflows below.

Writing Operations

Similar to Features, Operations group multiple Jobs that are often executed together, or their combination form a single piece of functionality that itself can be split into multiple reusable steps. However, they differ in testing.

  1. Generate an Operation class, which will create its test class along.

  2. Open the Operation class and describe the steps that implement the functionality in comments

 class NotifyFollowersOperation extends Operation
 {
     public function handle(Request $request)
     {
         // 1. @TODO: query followers IDs (paginate)

         // 2. @TODO: enqueue notifications in batches of 50

         // 3. @TODO: return status
     }
 }

  1. Start writing the test for the operation: It is a unit test to ensure one thing:

That it runs the correct jobs with the correct parameters, in the correct order.

Jobs don't really run within an Operation test, we rely on the unit test of the single job to ensure its integrity, while operations are about running multiple jobs in sequence. We achieve this by partially mocking the Operation class.

 $mOperation = Mockery::mock(NotifyFollowersOperation::class)->makePartial();

And then we ensure it runs and returns as expected:

 $mOperation->shouldReceive('run')
     ->with(GetFollowersJob::class, ['userId' => $userId, 'fields' => 'id'])
     ->once()->andReturn($result);

With this we have drawn a map of how we would like our codebase to be composed on the outside, and what is expected from each component for this implementation. It is now that we start detecting reusable components and we adjust as desired. For example we might figure that a similar functionality is already implemented and needs a simple adjustment to be used in the current operation or feature, so we go ahead and include a @TODO item to update it.

Of course, this is an iterative process where we rarely get things right from the first time so feel free to draft as many possibilities until you are satisfied with the final result, then start filling Job classes with implementation.

At this stage we would have had to create our Job classes so that we cloud use them in the test of our Operation and Feature classes, next will be to fill those Jobs with implementation.

Writing Jobs

Resulting from our previous steps of the workflow, a list of tasks that we can start ticking off and our feature will be implemented completely by the end. All that is left is to start filling Job classes with implementations.

  1. If you hadn't generated the Job class previously, generate a Job class for each of the items previously marked as a "to do" item. It is usually best to generate one class at a time, fill it, ensure that it works as expected and that it provides the functionality required and then move on to the next. As for the sequence of generating the jobs, it is usually best to follow the sequence in the Feature class.

  2. Open the generated Job class and start writing functionality. If you are into TDD, you may prefer to open the test file and write your expectations first. Nevertheless, it is not important nor does it affect the outcome to whether you write the test or the implementation first; whichever suites you best.

Demo

Let's exercise this workflow with a demo. If you prefer to read the resulting code you can find it on https://github.com/lucid-architecture/demo-workflow

To simplify this exercise, we will be working with a Lucid Microservice installation.

Create project

 composer create-project lucid-arch/laravel-microservice workflow-demo

Generate Feature

lucid make:feature UpdateArticle

Fill the Feature preliminarily to know what steps would fulfil its functionality

app/Features/UpdateArticleFeature.php
namespace App\Features;

use Lucid\Foundation\Feature;
use Illuminate\Http\Request;

class UpdateArticleFeature extends Feature
{
    public function handle(Request $request)
    {
        // 1. @TODO: validate request input (format & length)

        // 2. @TODO: verify access to resource

        // 3. @TODO: update resource

        // 4. @TODO: save audit activity (async)

        // 5. @TODO: notify followers (async)

        // 6. @TODO: respond with update status
    }
}

And now we tackle these jobs and operations one by one.

Step 1 - Validate Input

// 1. @TODO: validate request input (format & length)

Generate the class with lucid make:job ValidateArticleInput Article and fill it with the expected content

No we write the code and the test for our first job.

Switch between Code and Test in the tabs below. This applies to all classes with tests.

app/Domains/Article/Jobs/ValidateArticleInputJob.php
namespace App\Domains\Article\Jobs;

use Lucid\Foundation\Job;
use Lucid\Foundation\Validator;

class ValidateArticleInputJob extends Job
{
    /**
     * @var array
     */
    private $input;

    /**
     * @var array
     */
    private $rules = [
        'title' => ['required', 'string', 'max:100'],
        'content' => ['required', 'string', 'max:1000'],
    ];

    /**
     * Create a new job instance.
     * @param array $input
     */
    public function __construct(array $input)
    {
        $this->input = $input;
    }

    /**
     * Execute the job.
     *
     * @param Validator $validator
     * @return bool
     *
     * @throws \Lucid\Foundation\InvalidInputException
     */
    public function handle(Validator $validator) : bool
    {
        return $validator->validate($this->input, $this->rules);
    }
}

Running the tests now will yield the following results

PHPUnit 7.5.8 by Sebastian Bergmann and contributors.

.......I..                                                        10 / 10 (100%)

Time: 351 ms, Memory: 18.00 MB

OK, but incomplete, skipped, or risky tests!
Tests: 10, Assertions: 9, Incomplete: 1.

I know, the incomplete test is annoying, but it is there for a purpose! This is the Feature test, telling us that we're not done yet. Lucid test files are initially generated with a markTestIncomplete for that purpose.

Step 2 - Authorise Access

// 2. @TODO: verify access to resource

Generate Job with lucid make:job AuthoriseArticleUpdateJob Article

This job will require some setup before we can dive into it since it deals with database. We will use SQLite for testing, which will work as-is with any other SQL database, thanks to Eloquent.

Update phpunit.xml to run the tests in an SQLite memory instance

...
        <server name="DB_CONNECTION" value="sqlite"/>
        <server name="DB_DATABASE" value=":memory:"/>
    </php>
</phpunit>

A fix is required for factories to work, go to database/factories/ModelFactory.php and change the namespace of User::class to App\Data\Models\User::class

Generate a migration to create our Article model's table

php artisan make:migration create_articles_table

Fill the migration as follows

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateArticlesTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('articles', function (Blueprint $table) {
            $table->bigIncrements('id');
            $table->timestamps();
            $table->string('title');
            $table->text('content');
            $table->unsignedBigInteger('user_id');

            $table->foreign('user_id')
                ->references('id')->on('users')
                ->onDelete('cascade');
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('articles');
    }
}

Now we can create our Article model. Create app/Data/Models/Article.php

app/Data/Models/Article.php
namespace App\Data\Models;

use Illuminate\Database\Eloquent\Model;

class Article extends Model
{
    protected $fillable = ['title', 'content'];

    public function user()
    {
        return $this->belongsTo(User::class, 'user_id');
    }
}

Add the relationship User <> Article

public function articles()
{
    return $this->hasMany(Article::class, 'user_id');
}

Create Article factory to help us with our tests

database/factories/ArticleFactory.php
$factory->define(App\Data\Models\Article::class, function (Faker\Generator $faker) {
    return [
        'title' => $faker->title,
        'content' => $faker->realText,
        'user_id' => function () {
            return factory(App\Data\Models\User::class)->create()->id;
        },
    ];
});

Now we are ready to start writing our job.

This job simply fetches the user with the intended article as a relationship with a firstOrFail which will throw a ModelNotFoundException if the record is not found, indicating that either one of the records doesn't exist (user or article) or the relationship between them doesn't exist, meaning that the intended article doesn't belong to this user. Then we write the test to ensure this functionality serves as intended.

app/Domains/Article/Jobs/AuthoriseArticleUpdateJob.php
namespace App\Domains\Article\Jobs;

use Lucid\Foundation\Job;
use App\Data\Models\User;
use Illuminate\Database\Eloquent\ModelNotFoundException;

class AuthoriseArticleUpdateJob extends Job
{
    /**
     * @var User
     */
    private $user;
    /**
     * @var int
     */
    private $articleId;

    /**
     * Create a new job instance.
     *
     * @param User $user
     * @param int $articleId
     */
    public function __construct(User $user, int $articleId)
    {
        $this->user = $user;
        $this->articleId = $articleId;
    }

    /**
     * Execute the job.
     *
     * @throws ModelNotFoundException
     */
    public function handle()
    {
        // only the owner of the article is authorised to update it.
        User::where('id', $this->user->id)
                    ->whereHas('articles', function ($query) {
            $query->where('id', $this->articleId);
        })->firstOrFail();
    }
}

Step 3 - Save Article

// 3. @TODO: update resource

Generate Job lucid make:job SaveArticle Article

Saving an article is simple, we only need its identifier and the new attributes.

app/Domains/Article/Jobs/SaveArticleJob.php
namespace App\Domains\Article\Jobs;

use Lucid\Foundation\Job;
use App\Data\Models\Article;

class SaveArticleJob extends Job
{
    /**
     * @var string
     */
    private $title;

    /**
     * @var string
     */
    private $content;
    /**
     * @var int
     */
    private $articleId;

    /**
     * Create a new job instance.
     *
     * @param int $articleId
     * @param string $title
     * @param string $content
     */
    public function __construct(int $articleId, string $title, string $content)
    {
        $this->title = $title;
        $this->content = $content;
        $this->articleId = $articleId;
    }

    /**
     * Execute the job.
     *
     * @return int The number of records that were changed.
     *              It will be 1 if update was successful.
     *              Otherwise it's a 0.
     */
    public function handle() : int
    {
        return Article::where('id', $this->articleId)->update([
            'title' => $this->title,
            'content' => $this->content,
        ]);
    }
}

Step 4 - Add Audit Entry Asynchronously

// 4. @TODO: save audit activity (async)

This is a fancy step that in most cases won't be needed, but it's been added to the workflow to showcase working with async using Laravel queues. Logging activity is simply adding a record to an activity database table, but adding activity should not be a blocking action for it is not required by the response to the user, to whom we should be responding as quick as possible.

Create activity table php artisan make:migration create_activity_table

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateActivityTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('activity', function (Blueprint $table) {
            $table->bigIncrements('id');
            $table->string('description');
            $table->unsignedBigInteger('user_id');
            $table->timestamps();

            $table->foreign('user_id')
                ->references('id')->on('users')
                ->onDelete('cascade');
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('activity');
    }
}

Create model app/Data/Models/ActivityLog.php

namespace App\Data\Models;

use Illuminate\Database\Eloquent\Model;

class ActivityLog extends Model
{
    protected $table = 'activity';

    protected $fillable = ['user_id', 'description'];

    public function actor()
    {
        return $this->belongsTo(User::class, 'user_id');
    }
}

Generate job lucid make:job LogActivity Activity --queue

You will notice that the generated job extends QueueableJob instead of Job which will instruct the dispatcher when calling $this->run(Job) in the Feature or Operation to run it asynchronously in Laravel queue as configured in the application.

app/Domains/Activity/Jobs/LogActivityJob.php
namespace App\Domains\Activity\Jobs;

use App\Data\Models\ActivityLog;
use Lucid\Foundation\QueueableJob;

class LogActivityJob extends QueueableJob
{
    /**
     * @var int
     */
    private $userId;

    /**
     * @var string
     */
    private $description;

    /**
     * Create a new job instance.
     *
     * @param int $userId
     * @param string $description
     */
    public function __construct(int $userId, string $description)
    {
        $this->userId = $userId;
        $this->description = $description;
    }

    /**
     * Execute the job.
     *
     * @return ActivityLog
     */
    public function handle() : ActivityLog
    {
        return ActivityLog::create([
            'user_id' => $this->userId,
            'description' => $this->description,
        ]);
    }
}

Step 5 - Notify Followers

// 5. @TODO: notify followers (async)

Notifying followers here is meant as if every follower of this author is subscribed to receive some sort of a notification every time this author publishes a new post. It is made of two steps:

  1. Fetching a user's followers

  2. Sending them notifications

Similar to our activity log, since it is not essential for the response to the user, we better do these steps asynchronously.

Since each of these steps should be in its own Job according to the Lucid principles, and so that we don't run two jobs every time we need to notify followers, Operations are handy in this scenario. But we will follow an outward approach where we start by creating the jobs and ensuring that they work as expected, then create the operation that runs them, just as we are doing with our Feature here. This approach is recommended because after finishing with the jobs we will know what to the Operation should expect as input parameters based on what the jobs require.

Create follows table migration php artisan:make migration create_followers_table

which will be our pivot table to link users with their followers.

use Illuminate\Support\Facades\Schema;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Database\Migrations\Migration;

class CreateFollowsTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('follows', function (Blueprint $table) {
            $table->unsignedBigInteger('author_id');
            $table->unsignedBigInteger('follower_id');
            $table->timestamps();

            $table->foreign('author_id')
                ->references('id')->on('users')
                ->onDelete('cascade');

            $table->foreign('follower_id')
                ->references('id')->on('users')
                ->onDelete('cascade');

        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('follows');
    }
}

Add relationship method and Notifiable trait to User model

class User extends Authenticatable
{
    use Notifiable;

    //...

        public function followers()
    {
        return $this->belongsToMany(self::class,
            'follows',
            'author_id',
            'follower_id')
            ->withTimestamps();
    }
}

Generate Job to fetch user followers lucid make:job GetUserFollowers User

In a real-life scenario you should paginate followers instead of fetching them all at once, this is done this way to keep this demo simple.

app/Domains/User/Jobs/GetUserFollowersJob.php
namespace App\Domains\User\Jobs;

use Lucid\Foundation\Job;
use App\Data\Models\User;
use Illuminate\Database\Eloquent\Collection;

class GetUserFollowersJob extends Job
{
    /**
     * @var int
     */
    private $userId;

    /**
     * Create a new job instance.
     *
     * @param int $userId
     */
    public function __construct(int $userId)
    {
        $this->userId = $userId;
    }

    /**
     * Execute the job.
     *
     * @return Collection of User models
     */
    public function handle() : Collection
    {
        return User::find($this->userId)->followers;
    }
}

Generate notification to be sent php artisan make:notification ArticlePublished

We will use this class as-is, no modifications are required here.

Generate Job to send notifications to users lucid make:job NotifyFollowers User

app/Domains/User/Jobs/NotifyFollowersJob.php
namespace App\Domains\User\Jobs;

use Lucid\Foundation\Job;
use App\Notifications\ArticlePublished;
use Illuminate\Database\Eloquent\Collection;
use Illuminate\Support\Facades\Notification;

class NotifyFollowersJob extends Job
{
    /**
     * @var Collection
     */
    private $followers;

    /**
     * Create a new job instance.
     *
     * @param Collection $followers of User models
     */
    public function __construct(Collection $followers)
    {
        $this->followers = $followers;
    }

    /**
     * Execute the job.
     */
    public function handle()
    {
        Notification::send($this->followers, new ArticlePublished());
    }
}

Generate Operation to run both jobs lucid make:operation NotifyFollowers --queue

And its test. The objective of testing operations is to ensure they run the correct jobs with the correct parameters in the correct sequence. It is not necessary to run the actual job because we would be testing the job twice while we should be keen not to reinvent the wheel nor replicating tests, we rely on the Job's tests to provide its guarantee.

app/Operations/NotifyFollowersOperation.php
namespace App\Operations;

use Lucid\Foundation\QueueableOperation;
use Illuminate\Http\Request;
use App\Domains\User\Jobs\NotifyFollowersJob;
use App\Domains\User\Jobs\GetUserFollowersJob;

class NotifyFollowersOperation extends QueueableOperation
{
    /**
     * @var int
     */
    private $userId;

    /**
     * NotifyFollowersOperation constructor.
     * @param int $userId
     */
    public function __construct(int $userId)
    {
        $this->userId = $userId;
    }

    public function handle()
    {
        $followers = $this->run(GetUserFollowersJob::class, [
            'userId' => $this->userId
        ]);

        $this->run(NotifyFollowersJob::class, compact('followers'));
    }
}

Step 6 - Respond with Update Status

Simply return a JSON response using built-in Http\RespondWithJsonJob as seen in the next step.

Step 7 - Finally: Compose Feature

Now that we have all the jobs ready and we're sure that they are working as expected through their unit tests, all we have to do now is run them in sequence in our Feature class and pass the correct parameters and everything should work as expected.

Create controller lucid make:controller Article --plain

Add update controller method to serve the feature, and route in routes/web.php

Route::put('/articles/{articleId}', 'ArticleController@update');

app/Http/Controllers/ArticleController.php
namespace App\Http\Controllers;

use Lucid\Foundation\Http\Controller;
use App\Features\UpdateArticleFeature;

class ArticleController extends Controller
{
    public function update($articleId)
    {
        return $this->serve(UpdateArticleFeature::class, 
            ['articleId' => intval($articleId)]);
    }
}

Fill the feature's handle method and test it.

app/Features/UpdateArticleFeature.php
namespace App\Features;

use Auth;
use Lucid\Foundation\Feature;
use Illuminate\Http\Request;
use App\Domains\Article\Jobs\SaveArticleJob;
use App\Operations\NotifyFollowersOperation;
use App\Domains\Activity\Jobs\LogActivityJob;
use App\Domains\Http\Jobs\RespondWithJsonJob;
use App\Domains\Article\Jobs\ValidateArticleInputJob;
use App\Domains\Article\Jobs\AuthoriseArticleUpdateJob;

class UpdateArticleFeature extends Feature
{
    /**
     * @var int
     */
    private $articleId;

    /**
     * CreateArticleFeature constructor.
     *
     * @param int $articleId
     */
    public function __construct(int $articleId)
    {
        $this->articleId = $articleId;
    }

    /**
     * @param Request $request
     */
    public function handle(Request $request)
    {
        $this->run(new ValidateArticleInputJob($request->input()));

        $this->run(AuthoriseArticleUpdateJob::class, [
            'user' => Auth::user(),
            'articleId' => $this->articleId,
        ]);

        $isSuccess = $this->run(SaveArticleJob::class, [
            'articleId' => $this->articleId,
            'title'=> $request->input('title'),
            'content' => $request->input('content'),
        ]);

        $this->run(LogActivityJob::class, [
            'userId' => Auth::user()->id,
            'description' => "Article updated ".$this->articleId,
        ]);

        $this->run(NotifyFollowersOperation::class, [
            'userId' => Auth::user()->id,
        ]);

        return $this->run(new RespondWithJsonJob(['success' => $isSuccess]));
    }
}

Notable Observation

Through this workflow, it is important to note how tests cover the scope of the component only and keeps us from breaking the boundaries where we shouldn't. Especially when it comes to the feature test where it is kept to what it is responsible for, without having to repeat the tests that's been covered by the jobts themselves. However, it might be important to test edge cases at the feature level to ensure that the responses from our backend are consistent and as expected.

FAQ

Why sometimes we use UserId and other times the whole User?

Because of the scope. For example: AuthoriseArticleUpdateJob might require more information about the user in the future than their ID because the nature of what it does - authorisation. While other jobs such as LogActivityJob and NotifyFollowersOperation are only concerned about the user's identifier, regardless of where and how this was retrieved.

Doing this allows for the reuse of these jobs beyond the scope of this feature alone. We can use them anywhere else (for example from a command where users aren't authenticated) without having to workaround authentication where it doesn't belong.

Could we have improved logging activity?

Yes. For example, create a job for each activity (i.e. LogArticleUpdateActivityJob instead of a generic LogActivityJob. It would be preferred over having a string in the feature which is recommended to be avoided when possible but kept here for the simplicity of this exercise.

Can we add more safety to this code?

Yes, plenty! A simple addition would be to ensure there's a session before proceeding with the feature to avoid errors such as:

Symfony\Component\Debug\Exception\FatalThrowableError: Argument 1 passed to 
App\Domains\Article\Jobs\AuthoriseArticleUpdateJob::__construct() 
must be an instance of App\Data\Models\User, null given

This could be done around the HTTP Kernel like in the middleware area.

Another safety check would be to wrap async calls with a try/catch block so that failures in there don't affect the user's response (namely LogActivityJob and NotifyFollowersOperation because they do not dictate the actual status of the update.

PreviousPart INextBest Practices

Last updated