I was recently asked to add Blacklight Advanced Search to a Hyku app for the US Department of Transportation. It was a little tricky, so I’m documenting the process in the hopes of making life easier for the next person who has to do this.

Many thanks to Dean Farrell at UNC Libraries for pointing me at UNC’s hy-c implementation, where blacklight advanced search is installed into Hyrax: https://github.com/UNC-Libraries/hy-c/pull/421/files

David Kinzer’s Blacklight Search Notes are also excellent background reading: https://gist.github.com/dkinzer/4f6dbb4634dbbdc99255dbea6305ccae

Write a feature spec first

As always, start with a test. I like to write a high-level feature test as a way of keeping myself focused on what I’m trying to deliver, and then I fill in unit tests for specific methods as needed. Here is my nearly-empty high level feature test. Note that I don’t really know what advanced search is going to look like yet, so I’m not spending lots of time on detail yet. However, it’s still helpful to have this test first. It gives me a way to quickly iterate and prove to myself that the changes I’m making are moving me in the right direction, and it lets me build the test up over time as I’m implementing the feature.

# frozen_string_literal: true

require 'rails_helper'
include Warden::Test::Helpers

RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do
  context 'an unauthenticated user' do
    scenario 'advanced search basic sanity check' do
      visit '/advanced'
      fill_in('Title', with: 'ambitious aardvark')
      find('#advanced-search-submit').click
      expect(page).to have_content('ambitious aardvark')
      expect(page).to have_content('No results found for your search')
    end
  end
end

Installing blacklight_advanced_search

I follow the blacklight_advanced_search basic instructions:

Add to your application's Gemfile:

gem "blacklight_advanced_search"

then run 'bundle install'. Then run:

rails generate blacklight_advanced_search

Note: It will offer to generate a new basic search partial for you. You do NOT want it. It will break the basic search in Hyku. If you end up with a new app/views/catalog/_search_form.html.erb just delete it and add by hand anywhere you want a link to advanced search.

So, I follow the installation instructions, and then I run my feature spec. The first time I do, I get this error:

undefined method facets_for_advanced_search_form for Hyrax::CatalogSearchBuilder (NoMethodError)`

So, clearly, I’m missing some configuration.

Tracing the error

I drop a byebug in at the line indicated by the stack trace. In this case: blacklight-6.23.0/lib/blacklight/search_builder.rb:147

[142, 151] in /usr/local/rvm/gems/ruby-2.5.8/gems/blacklight-6.23.0/lib/blacklight/search_builder.rb
   142:     #
   143:     # @return a params hash for searching solr.
   144:     def processed_parameters
   145:       request.tap do |request_parameters|
   146:     byebug
=> 147:         processor_chain.each do |method_name|
   148:           send(method_name, request_parameters)
   149:         end
   150:       end
   151:     end
(byebug) request_parameters
{"facet.field"=>[], "facet.query"=>[], "facet.pivot"=>[], "fq"=>[], "hl.fl"=>[]}
(byebug) processor_chain
[:default_solr_parameters, :add_query_to_solr, :add_facet_fq_to_solr, :add_facetting_to_solr, :add_solr_fields_to_query, :add_paging_to_solr, :add_sorting_to_solr, :add_group_config_to_solr, :add_facet_paging_to_solr, :add_access_controls_to_solr_params, :filter_models, :only_active_works, :add_access_controls_to_solr_params, :show_works_or_works_that_contain_files, :show_only_active_records, :filter_collection_facet_for_access, :facets_for_advanced_search_form]

That : facets_for_advanced_search_form at the end of the processor chain seems like a likely culprit, so I’m going to remove it and see if that gets my form to render. Initially, I’m just going to add a next right in place, to see if this fixes the problem, before I go to the trouble of figuring out the right way to do it:

    def processed_parameters
      request.tap do |request_parameters|
        processor_chain.each do |method_name|
          next if method_name == :facets_for_advanced_search_form
          send(method_name, request_parameters)
        end
      end
    end

And now my form renders!

So I’ve proven to myself that the problem is this missing facets_for_advanced_search_form method, but how to fix that in a maintainable way?

Override as little as possible to keep local code maintainable

Let’s go look at the class mentioned in the error message: Hyrax::CatalogSearchBuilder

Here is the source of that method: https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/app/search_builders/hyrax/catalog_search_builder.rb

Notice the excellent guidance provided here:

# If you plan to customize the base catalog search builder behavior (e.g. by
# adding a mixin module provided by a blacklight extension gem), inheriting this
#  class, customizing behavior, and reconfiguring `CatalogController` is the
# preferred mechanism.

Sounds good to me. Let’s try that!

So, I’m going to start by writing a test, of course. I go look at how the search builders are set up in hyrax, and I make this very basic test in spec/search_builders/ntl_search_builder_spec.rb (note that this is adapted from the test setup at https://github.com/samvera/hyrax/blob/130c4e600318a9194725b35dd0fd6e19e5108dd9/spec/search_builders/hyrax/collection_search_builder_spec.rb… I did not attempt to write this off the top of my head.)

# frozen_string_literal: true
RSpec.describe NtlSearchBuilder do
  let(:scope) do
    double(blacklight_config: CatalogController.blacklight_config,
           current_ability: ability)
  end
  let(:user) { create(:user) }
  let(:ability) { ::Ability.new(user) }
  let(:access) { :read }
  let(:builder) { described_class.new(scope).with_access(access) }
  it "can be instantiated" do
    expect(builder).to be_instance_of(described_class)
  end
end

This test doesn’t do anything except verify that the class is set up correctly, but that is better than nothing, and it gives us a place where we can flesh out behavior as we need it. I run it, and of course it fails, for the expected reasons: NameError: uninitialized constant NtlSearchBuilder

I then define the class, by making this file at app/search_builders/ntl_search_builder.rb:

class NtlSearchBuilder < Hyrax::CatalogSearchBuilder

end

And now my test passes!

Now, we set CatalogController to use this new class:

    # Use locally customized NtlSearchBuilder so we can enable blacklight_advanced_search
    config.search_builder_class = NtlSearchBuilder

And I add the facets_for_advanced_search_form method to my NtlSearchBuilder class, just copying the method (https://github.com/projectblacklight/blacklight_advanced_search/blob/master/lib/blacklight_advanced_search/advanced_search_builder.rb#L77-L96)

I also add the lines that the Blacklight advanced search generator installed into app/models/search_builder.rb. This is the bit that actually combines the Advanced Search query capabilities with the Hyrax provided search capabilities (e.g., search gated by permissions, workflow, group membership, etc.)

Now my class looks like this:

##
# A locally defined search builder, which will allow us to customize the search
# behavior of this application. In particular, this is needed to allow us to
# use blacklight_advanced_search.
class NtlSearchBuilder < Hyrax::CatalogSearchBuilder
  include Blacklight::Solr::SearchBuilderBehavior
  include BlacklightAdvancedSearch::AdvancedSearchBuilder
  self.default_processor_chain += [:add_advanced_parse_q_to_solr, :add_advanced_search_to_solr]


  # A Solr param filter that is NOT included by default in the chain,
   # but is appended by AdvancedController#index, to do a search
   # for facets _ignoring_ the current query, we want the facets
   # as if the current query weren't there.
   #
   # Also adds any solr params set in blacklight_config.advanced_search[:form_solr_parameters]
   def facets_for_advanced_search_form(solr_p)
     # ensure empty query is all records, to fetch available facets on entire corpus
     solr_p["q"]            = '{!lucene}*:*'
     # explicitly use lucene defType since we are passing a lucene query above (and appears to be required for solr 7)
     solr_p["defType"]      = 'lucene'
     # We only care about facets, we don't need any rows.
     solr_p["rows"]         = "0"

     # Anything set in config as a literal
     if blacklight_config.advanced_search[:form_solr_parameters]
       solr_p.merge!(blacklight_config.advanced_search[:form_solr_parameters])
     end
   end
end

And now my advanced search page renders and my feature test for advanced search passes.

One last step: a more robust feature test with real data

One last step: I flesh out my advanced search feature spec a bit, including loading some actual client provided data, ensuring the advanced search feature really does work as expected. Here is the feature spec with some of that added:

# frozen_string_literal: true

require 'rails_helper'
include Warden::Test::Helpers

RSpec.feature 'Advanced Search', type: :feature, js: true, clean: true do
  context 'empty solr index' do
    scenario 'basic search sanity check' do
      visit '/'
      fill_in('q', with: 'ambitious aardvark')
      click_button('Go')
      expect(page).to have_content('ambitious aardvark')
      expect(page).to have_content('No results found for your search')
    end
    scenario 'advanced search basic sanity check' do
      visit '/advanced'
      fill_in('Title', with: 'ambitious aardvark')
      find('#advanced-search-submit').click
      expect(page).to have_content('ambitious aardvark')
      expect(page).to have_content('No results found for your search')
    end
  end

  ##
  # Load solr data from Department of Transportation import testing and ensure advanced
  # search works reasonably well with this data.
  context 'with data' do
    let!(:admin_set_collection_type) { FactoryBot.create(:admin_set_collection_type) }
    let!(:user_collection_type) { FactoryBot.create(:user_collection_type) }

    before do
      solr = Blacklight.default_index.connection
      sample_records = JSON.parse(File.open(File.join(fixture_path, 'solr', 'dot-sample-data.json')).read)
      docs = sample_records["response"]["docs"]
      docs.each do |doc|
        doc.delete("_version_")
        doc.delete("score")
        solr.add(doc)
      end
      solr.commit
    end
    scenario 'basic search sanity check' do
      visit '/'
      click_button('Go')
      number_of_search_results = find('span.page_entries').find_all('strong').last.text
      expect(number_of_search_results).to eq('17')
    end
    it "searches by title" do
      visit '/advanced'
      fill_in('Title', with: 'habitat')
      find('#advanced-search-submit').click
      number_of_search_results = find('span.page_entries').find_all('strong').last.text
      expect(number_of_search_results).to eq('1')
    end
  end
end

I recently had to find a bug in a Hyrax application, and I thought it might be helpful if I documented the process.

1. Define the problem

I like to start a bug hunt with a clear description of what exactly I’m trying to solve. A great place to do this is in a ticket on the team’s board. In this case, the problem was that when new works were submitted to Hyrax via the create work form, the visibility was not being persisted. I find it helpful to write out the bug definition as a user story:

As a content contributor, when I submit a new work through the form, I want to be able to make it public so that it will be visible to the world. Instead, the work always ends up being marked as restricted / private. 

2. Write a test. Or several.

I wrote a test to check whether the issue was with visibility in general, or whether it was specific to form submission. One big question for me with this bug is whether the problem is with the form submission or the object creation process, so I first wrote a test to see what happens when we create an object without the form.

I like FactoryBot for writing tests, and in a Hyrax app I always make a Hyrax::UploadedFile factory like this:

FactoryBot.define do
  factory :uploaded_file, class: Hyrax::UploadedFile do
    file { Rack::Test::UploadedFile.new("#{::Rails.root}/spec/fixtures/image.jp2") }
    user_id { 1 }
  end
end

And here is my test to sanity check the actor stack, to ensure that, assuming all the parameters reach it, it will behave as expected. This test passed, which tells me that the problem isn’t in the actor stack:

require 'rails_helper'
include Warden::Test::Helpers

RSpec.describe 'Sanity check the actor stack', type: feature, js: false, clean: true do
  context 'a new object created without using the submission form' do
    let(:user) { FactoryBot.create(:user) }
    let(:article) { FactoryBot.build(:article, depositor: user.user_key, visibility: "open") }
    let(:uploaded_file) { FactoryBot.create(:uploaded_file, user_id: user.id)}
    let(:attributes_for_actor) { { uploaded_files: [uploaded_file.id] } }
    it "saves the expected visibility" do
        env = Hyrax::Actors::Environment.new(article, ::Ability.new(user), attributes_for_actor)
        Hyrax::CurationConcern.actor.create(env)
        expect(Article.count).to eq 1
        post_actor_stack_article = Article.last
        expect(post_actor_stack_article.visibility).to eq "open"
    end
  end
end

So, interesting: I haven’t been able to write a failing test for this bug yet. Let’s try again, this time using the form:

# Generated via
#  `rails generate hyrax:work Article`
require 'rails_helper'
include Warden::Test::Helpers

# NOTE: If you generated more than one work, you have to set "js: true"
RSpec.describe 'Create a Article', type: feature, js: true, clean: true do
  context 'a logged in user' do
    let(:user_attributes) do
      { email: 'test@example.com' }
    end
    let(:user) do
      User.new(user_attributes) { |u| u.save(validate: false) }
    end
    let(:admin_set_id) { AdminSet.find_or_create_default_admin_set_id }
    let(:permission_template) { Hyrax::PermissionTemplate.find_or_create_by!(source_id: admin_set_id) }
    let(:workflow) { Sipity::Workflow.create!(active: true, name: 'test-workflow', permission_template: permission_template) }

    before do
      # Create a single action that can be taken
      Sipity::WorkflowAction.create!(name: 'submit', workflow: workflow)

      # Grant the user access to deposit into the admin set.
      Hyrax::PermissionTemplateAccess.create!(
        permission_template_id: permission_template.id,
        agent_type: 'user',
        agent_id: user.user_key,
        access: 'deposit'
      )
      login_as user
    end

    it do
      visit '/dashboard'
      click_link "Works"
      click_link "Add new work"

      # If you generate more than one work uncomment these lines
      choose "payload_concern", option: "Article"
      click_button "Create work"

      expect(page).to have_content "Add New Article"
      click_link "Files" # switch tab
      expect(page).to have_content "Add files"
      expect(page).to have_content "Add folder"
      within('span#addfiles') do
        attach_file("files[]", Rails.root.join('spec/fixtures/image.jp2'), visible: false)
        attach_file("files[]", Rails.root.join('spec/fixtures/jp2_fits.xml'), visible: false)
      end
      click_link "Descriptions" # switch tab
      fill_in('Title', with: 'My Test Work')
      fill_in('Creator', with: 'Doe, Jane')
      fill_in('Description', with: 'A brief description of my article')
      select('In Copyright', from: 'article_rights_statement')

      # With selenium and the chrome driver, focus remains on the
      # select box. Click outside the box so the next line can't find
      # its element
      find('body').click
      select('Article', from: 'article_resource_type')

      # With selenium and the chrome driver, focus remains on the
      # select box. Click outside the box so the next line can't find
      # its element
      find('body').click
      choose('article_visibility_open')
      expect(page).to have_content('Please note, making something visible to the world (i.e. marking this as Public) may be viewed as publishing which could impact your ability to')
      check('agreement')

      click_on('Save')
      expect(page).to have_content('My Test Work')
      expect(page).to have_content "Your files are being processed by Hyrax in the background."
      expect(Article.count).to eq 1
      article = Article.last
      expect(article.visibility).to eq "open"
    end
  end
end

If you do any Hyrax development, you’ll notice that this test is mostly the Hyrax-generated feature spec, with a few small tweaks. I had already been spending time ensuring that the feature specs were running for this project, so I had already spent some time tweaking this and I knew it was green. All I added were the last two lines:

      article = Article.last
      expect(article.visibility).to eq "open"

And this test FAILS! Hooray, a test that fails for the right reasons. I have isolated my bug and I now have clear criteria for what it means for this bug to be fixed, along with a check to ensure it doesn’t creep back in as a regression. Now, let’s actually find the bug.

3. The bug hunt

My primary tool on a bug hunt is the byebug command. I drop it into various places in the code, run my test, and see if I hit my byebug. Once I’m in my byebug console, I poke around to see what I can see. If everything looks normal, I move on.

First stop: WorksControllerBehavior

On our tour of what happens to the data after it is submitted via the Hyrax form, we pass briefly through our locally generated controller. We’ve just submitted an article, so the controller in question is app/controllers/hyrax/articles_controller.rb. However, upon examination it’s clear there isn’t much there. Our local controller gives us a place to put overrides that might be specific to that Work type, but unless we’ve customized it most of its behavior is going to come via an include:

    include Hyrax::WorksControllerBehavior

I’m looking for a create method into which to drop a byebug, so I go to find the copy of Hyrax::WorksControllerBehavior that my test is executing. In this case, because we use a docker container development envionment at Notch8, the process looks like this:

> bess@lorca > docker-compose exec web bash
root@bc6de9202513:/data# bundle show hyrax
/usr/local/bundle/gems/hyrax-2.9.0
root@bc6de9202513:/data# cd /usr/local/bundle/gems/hyrax-2.9.0
root@bc6de9202513:/usr/local/bundle/gems/hyrax-2.9.0# vi app/controllers/concerns/hyrax/works_controller_behavior.rb

I find the create method, and drop a byebug in at the beginning. I run my test again and I see:

[51, 60] in /usr/local/bundle/gems/hyrax-2.9.0/app/controllers/concerns/hyrax/works_controller_behavior.rb
   51:       build_form
   52:     end
   53:
   54:     def create
   55:      byebug
=> 56:       if actor.create(actor_environment)
   57:         after_create_response
   58:       else
   59:         respond_to do |wants|
   60:           wants.html do
(byebug) params
<ActionController::Parameters {"utf8"=>"✓", "article"=>{"title"=>["My Test Work"], "creator"=>["Doe, Jane"], "description"=>["A brief description of my article"], "resource_type"=>["", "Article"], "rights_statement"=>"http://rightsstatements.org/vocab/InC/1.0/", "abstract"=>"", "bibliographic_citation"=>[""], "contributor"=>[""], "date_created"=>[""], "date_issued"=>"", "extent"=>[""], "funder"=>[""], "funder_identifier"=>[""], "grant_award"=>[""], "grant_number"=>[""], "grant_uri"=>[""], "identifier"=>[""], "institution_organization"=>[""], "issue"=>"", "keyword"=>[""], "language"=>[""], "license"=>[""], "note"=>[""], "peer_review_status"=>"", "publisher"=>[""], "related_resource"=>[""], "rights_notes"=>[""], "school"=>[""], "source"=>[""], "subject"=>[""], "title_alternative"=>[""], "volume"=>"", "admin_set_id"=>"admin_set/default", "member_of_collection_ids"=>"", "visibility"=>"open", "visibility_during_embargo"=>"restricted", "embargo_release_date"=>"2020-10-24", "visibility_after_embargo"=>"open", "visibility_during_lease"=>"open", "lease_expiration_date"=>"2020-10-24", "visibility_after_lease"=>"restricted"}, "uploaded_files"=>["1", "2"], "new_group_name_skel"=>"Select a group", "new_group_permission_skel"=>"none", "new_user_name_skel"=>"", "new_user_permission_skel"=>"none", "agreement"=>"1", "locale"=>"en", "controller"=>"hyrax/articles", "action"=>"create"} permitted: false>

So, first question answered: The visibility IS being sent from the form, and at the time it enters the actor stack on line 56 visibility does indeed equal open.

Time to dive into the actor stack.

A quick tour of the actor stack

Here is a quick way to find out what actors your object will traverse in the actor stack:

(byebug) Hyrax::CurationConcern.actor
#<Hyrax::Actors::TransactionalRequest:0x0000557a92022148 @next_actor=#<Hyrax::Actors::OptimisticLockValidator:0x0000557a92022170 @next_actor=#<Hyrax::Actors::CreateWithRemoteFilesActor:0x0000557a92022198 @next_actor=#<Hyrax::Actors::CreateWithFilesActor:0x0000557a920221c0 @next_actor=#<Hyrax::Actors::CollectionsMembershipActor:0x0000557a920221e8 @next_actor=#<Hyrax::Actors::AddToWorkActor:0x0000557a92022210 @next_actor=#<Hyrax::Actors::AttachMembersActor:0x0000557a92022238 @next_actor=#<Hyrax::Actors::ApplyOrderActor:0x0000557a92022260 @next_actor=#<Hyrax::Actors::DefaultAdminSetActor:0x0000557a92022288 @next_actor=#<Hyrax::Actors::InterpretVisibilityActor:0x0000557a920222b0 @next_actor=#<Hyrax::Actors::TransferRequestActor:0x0000557a920222d8 @next_actor=#<Hyrax::Actors::ApplyPermissionTemplateActor:0x0000557a92022300 @next_actor=#<Hyrax::Actors::CleanupFileSetsActor:0x0000557a92022328 @next_actor=#<Hyrax::Actors::CleanupTrophiesActor:0x0000557a92022350 @next_actor=#<Hyrax::Actors::FeaturedWorkActor:0x0000557a92022378 @next_actor=#<Hyrax::Actors::ModelActor:0x0000557a920223a0 @next_actor=#<Hyrax::Actors::InitializeWorkflowActor:0x0000557a920223c8 @next_actor=#<Hyrax::Actors::Terminator:0x0000557a92022468>>>>>>>>>>>>>>>>>>

Let’s break that out into something easier to read:

1. Hyrax::Actors::TransactionalRequest
2. Hyrax::Actors::OptimisticLockValidator
3. Hyrax::Actors::CreateWithRemoteFilesActor
4. Hyrax::Actors::CreateWithFilesActor
5. Hyrax::Actors::CollectionsMembershipActor
6. Hyrax::Actors::AddToWorkActor
7. Hyrax::Actors::AttachMembersActor
8. Hyrax::Actors::ApplyOrderActor
9. Hyrax::Actors::DefaultAdminSetActor
10. Hyrax::Actors::InterpretVisibilityActor
11. Hyrax::Actors::TransferRequestActor
12. Hyrax::Actors::ApplyPermissionTemplateActor
13. Hyrax::Actors::CleanupFileSetsActor
14. Hyrax::Actors::CleanupTrophiesActor
15. Hyrax::Actors::FeaturedWorkActor
16. Hyrax::Actors::ModelActor
17. Hyrax::Actors::InitializeWorkflowActor
18. Hyrax::Actors::Terminator

I won’t go into all of those, and some of them are clearly not relevant to this use case. I’m going to drop into the first one that I think might be relevant for me, Hyrax::Actors::CreateWithRemoteFilesActor:

[11, 20] in /usr/local/bundle/gems/hyrax-2.9.0/app/actors/hyrax/actors/create_with_remote_files_actor.rb
   11:     class CreateWithRemoteFilesActor < Hyrax::Actors::AbstractActor
   12:       # @param [Hyrax::Actors::Environment] env
   13:       # @return [Boolean] true if create was successful
   14:       def create(env)
   15:        byebug
=> 16:         remote_files = env.attributes.delete(:remote_files)
   17:         next_actor.create(env) && attach_files(env, remote_files)
   18:       end
   19:
   20:       # @param [Hyrax::Actors::Environment] env

When I examine the params at this point in the stack, I see something interesting: visibility is gone.

(byebug) env.attributes
{"title"=>["My Test Work"], "abstract"=>nil, "bibliographic_citation"=>[], "contributor"=>[], "creator"=>["Doe, Jane"], "date_created"=>[], "date_issued"=>nil, "description"=>["A brief description of my article"], "extent"=>[], "funder"=>[], "funder_identifier"=>[], "grant_award"=>[], "grant_number"=>[], "grant_uri"=>[], "identifier"=>[], "institution_organization"=>[], "issue"=>nil, "keyword"=>[], "language"=>[], "license"=>[], "note"=>[], "peer_review_status"=>nil, "publisher"=>[], "related_resource"=>[], "resource_type"=>["Article"], "rights_notes"=>[], "rights_statement"=>"http://rightsstatements.org/vocab/InC/1.0/", "school"=>[], "source"=>[], "subject"=>[], "title_alternative"=>[], "volume"=>nil, "remote_files"=>[], "uploaded_files"=>["1", "2"]}

When I examine the nascent object that the actor stack is acting upon, I see that it is mostly empty, and has the default visibility value, which is restricted:

(byebug) env.curation_concern
#<Article id: nil, head: [], tail: [], depositor: nil, title: [], date_uploaded: nil, date_modified: nil, state: nil, proxy_depositor: nil, on_behalf_of: nil, arkivo_checksum: nil, owner: nil, date_migrated: nil, format: [], bibliographic_citation: [], date_issued: nil, extent: [], institution_organization: [], note: [], related_resource: [], rights_notes: [], title_alternative: [], abstract: nil, funder: [], funder_identifier: [], grant_award: [], grant_number: [], grant_uri: [], issue: nil, peer_review_status: nil, school: [], source: [], volume: nil, label: nil, relative_path: nil, import_url: nil, resource_type: [], creator: [], contributor: [], description: [], keyword: [], license: [], rights_statement: [], publisher: [], date_created: [], subject: [], language: [], identifier: [], based_near: [], related_url: [], access_control_id: nil, representative_id: nil, thumbnail_id: nil, rendering_ids: [], admin_set_id: nil, embargo_id: nil, lease_id: nil>
(byebug) env.curation_concern.visibility
"restricted"

So, by the time we reach Hyrax::Actors::CreateWithRemoteFilesActor our “open” visibility value is missing, so it never gets applied to our skeleton curation_concern, which keeps its default visibility value of restricted. Hmm… I need to look eariler in the process. I do the same check in Hyrax::Actors::TransactionalRequest and see that visibility is also missing there. Where is my missing attribute?

Permitted params

The above got me far enough that I was able to ask a concrete question in the #hyrax channel on Samvera slack, where Tom Johnson pointed me to this place where params get sanitized:

[265, 274] in /usr/local/bundle/gems/hyrax-2.9.0/app/controllers/concerns/hyrax/works_controller_behavior.rb
   265:
   266:       # Add uploaded_files to the parameters received by the actor.
   267:       def attributes_for_actor
   268:           byebug
   269:         raw_params = params[hash_key_for_curation_concern]
=> 270:         attributes = if raw_params
   271:                        work_form_service.form_class(curation_concern).model_attributes(raw_params)
   272:                      else
   273:                        {}
   274:                      end

And there, on line 271, is where visibility goes missing. But WHY????

So what’s happening here is that we’re using Hyrax::ArticleForm to define what the permitted parameters are to get submitted through the form:

(byebug) work_form_service.form_class(curation_concern)
Hyrax::ArticleForm

And if I go look at Hyrax::ArticleForm I see that self.terms has been redefined:

    self.terms = [
      :title,
      :abstract,
      :bibliographic_citation,
      ...

Instead of redefining self.terms here, the pattern I prefer would be to add any customized fields to it, like this example from the RepoCamp curriculum:

When I change Hyrax::ArticleForm to instead use a +=, thus keeping all of the fields that come pre-defined in Hyrax, my feature test now passes.

   self.terms += [
     :title,
     :abstract,
     :bibliographic_citation,
     ...

In summary

1. Write your tests first
2. Keep your end-to-end feature tests green when you’re doing development
3. When you customize forms in Hyrax don’t re-define the terms, add to it and subtract from it as you like, but you probably don’t want to lose all the terms Hyrax gives you out of the box.

Docker and Kubernetes are pathways to more efficient development cycles and smoother, more reliable deployment for your application. These two platforms work together, one building on the other for managing containerized applications that make them lightweight and stable, which are huge pluses for your business. If Docker is the set of big steal boxes you put your applications in to make transporting them easy, Kubernetes is the network of trucks, boats and cranes that get them into place for production.

Containers, Docker, and Kubernetes

Containers are stand-alone packages that bundle everything required to run an application: code, libraries, system configurations, and other dependencies. Containerization is the use of these packages for application deployment or development. This encapsulation allows applications and components in different containers to be isolated from each other and the host machine It solves the problem of “well it worked on my machine” by packaging all those dependencies and configurations with the application every time it’s run, no matter where. What works in Docker on your machine will work in Docker in production.

Containerization is more lightweight and less hard-ware intensive than running on virtual machines, which require a discreet operating system for each application. Subsequently, there can be significant cost savings to running containerized environments.

Docker is an open source project used by developers and system engineers to build, run, and deploy containerized applications in virtually any environment. It is especially useful for deployment to the cloud. It provides a sort of instruction manual, the Docker image, which includes everything needed to run an app, including code and dependencies, setting up the private file system that the isolated container interacts with. Docker is a means to getting more apps running on existing servers, and is especially useful at supporting a microservices architecture.

Kubernetes is a platform for management and deployment of containerized applications. From the outside, it can seem like conversations about containerization use “Docker” and “Kubernetes” almost interchangeably, and once upon a time this was closer to true than it is today. However they are not simply two brands of the same product. Docker is a platform for creating and running containers, Kubernetes is an orchestration system for containers, used for automated deployment and monitoring of containers. Docker offers Docker Swarm, which does manage much of the same territory as Kubernetes. However, Kubernetes has more or less won out as the industry standard. Other tools, like Rancher, have even abandoned their own implementations for a solely Kubernetes-powered orchestration.

Docker and Kubernetes both have the advantage of being open source products. Using Docker for containerization is a huge efficiency gain for your development process even if you opt not to deploy to a Kubernetes cluster.

Why Docker?

Containers offer portable, lightweight packages for easy deployment of applications. But the meaning of that and the list of reasons to containerize applications goes deeper. Portability means developers can build locally and easily deploy to the cloud, and the flexibility of containers allow even complex applications to benefit. Key to the lightweight property of containers is that many applications can operate on one machine with one operating system, eliminating the heavy setup of virtual machines with individual operating systems. Containerizing your application allows 4 to 6 times the number of server applications to be run on the same hardware. 

Scalability is another key attraction for containerization, allowing for greater ability to increase and distribute containers across multiple servers. The isolated nature of containers also allows you to upgrade or completely replace one without threatening the rest of the application. 

Docker helps you create and manage containers for your applications. It provides a pathway to smooth, frequent software delivery. Irregular or infrequent deployments open the door for security issues and require more overhead. High performing teams deploy often, and Dockerized applications allow easy and confident deployment. 

There are several noteworthy benefits to Dockerizing your applications:

1. Consistency. Docker containers are identical on any system, with the exact specifications stored in the Dockerfile. This guarantees all builds will run identically, affording confidence as well as ease of identifying problematic issues.
2. Security. Because Docker app components are isolated, many security issues remain contained.
3. Conflict Reduction. The use of separate containers helps reduce clashing dependencies, both within an applications’ components and across multiple applications.
4. Environment Compartmentalization. Docker gives the opportunity for separate containers for testing, development, and production and makes it easy to deploy each.
5. CI Support. Docker works with continuous integration tools, so that whenever code is updated, it pushes to Docker Hub and then deploys to production.
6. Faster Onboarding. Development in containerized environments provides a simplified training process for new team members.

Summary

Dockerizing your applications can get more applications running on the same hardware with lower overhead costs. It allows developers to easily create containerized applications that are ready to run, and makes management and deployment much more efficient.

Choosing the best approach to containerization for your products’ needs can be overwhelming. Full Kubernetes for app management can be overkill, and having an experienced system engineer set up your platform is vital to mitigate performance, security, and configuration issues. Notch8 containerizes applications for most of our projects and we have extensive experience implementing Docker and Kubernetes.

We’d be happy to discuss how these solutions can help your product on a more efficient path to success. Contact Us with questions for our experienced DevOps team.

Additional Resources:

• How to Get Started with Docker
• Getting Started with Kubernetes