Refactor collections_dir feature for consistency (#6685)

Merge pull request 6685

features/collections_dir.feature

@@ -0,0 +1,161 @@
+Feature: Collections Directory
+  As a hacker who likes to structure content without clutter
+  I want to be able to organize my collections under a single directory
+  And render them from there
+
+  Scenario: Custom collections_dir containing only posts
+    And I have a collections/_posts directory
+    And I have the following post within the "collections" directory:
+      | title         | date       | content         |
+      | Gathered Post | 2009-03-27 | Random Content. |
+    And I have a "_config.yml" file with content:
+    """
+    collections_dir: collections
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And I should see "Random Content." in "_site/2009/03/27/gathered-post.html"
+
+  Scenario: Rendered collection in custom collections_dir also containing posts
+    Given I have a collections/_puppies directory
+    And I have the following document under the "puppies" collection within the "collections" directory:
+      | title  | date       | content            |
+      | Rover  | 2007-12-31 | content for Rover. |
+    And I have a collections/_posts directory
+    And I have the following post within the "collections" directory:
+      | title         | date       | content         |
+      | Gathered Post | 2009-03-27 | Random Content. |
+    And I have a "_config.yml" file with content:
+    """
+    collections:
+      puppies:
+        output: true
+
+    collections_dir: collections
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And the "_site/puppies/rover.html" file should exist
+    And I should see "Random Content." in "_site/2009/03/27/gathered-post.html"
+
+  Scenario: Rendered collection in custom collections_dir with posts at the site root
+    Given I have a collections/_puppies directory
+    And I have the following document under the "puppies" collection within the "collections" directory:
+      | title  | date       | content            |
+      | Rover  | 2007-12-31 | content for Rover. |
+    And I have a _posts directory
+    And I have the following post:
+      | title        | date       | content         |
+      | Post At Root | 2009-03-27 | Random Content. |
+    And I have a "_config.yml" file with content:
+    """
+    collections:
+      puppies:
+        output: true
+
+    collections_dir: collections
+    """
+    When I run jekyll build
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And the "_site/puppies/rover.html" file should exist
+    And the "_site/2009/03/27/post-at-root.html" file should not exist
+    And the _site/_posts directory should not exist
+
+  Scenario: Rendered collection in custom collections_dir also containing drafts
+    Given I have a collections/_puppies directory
+    And I have the following document under the "puppies" collection within the "collections" directory:
+      | title  | date       | content            |
+      | Rover  | 2007-12-31 | content for Rover. |
+    And I have a collections/_drafts directory
+    And I have the following draft within the "collections" directory:
+      | title          | date       | content         |
+      | Gathered Draft | 2009-03-27 | Random Content. |
+    And I have a "_config.yml" file with content:
+    """
+    collections:
+      puppies:
+        output: true
+
+    collections_dir: collections
+    """
+    When I run jekyll build --drafts
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And the "_site/puppies/rover.html" file should exist
+    And I should see "Random Content." in "_site/2009/03/27/gathered-draft.html"
+    And the _site/collections directory should not exist
+
+  Scenario: Rendered collection in custom collections_dir with drafts at the site root
+    Given I have a collections/_puppies directory
+    And I have the following document under the "puppies" collection within the "collections" directory:
+      | title  | date       | content            |
+      | Rover  | 2007-12-31 | content for Rover. |
+    And I have a _drafts directory
+    And I have the following draft:
+      | title         | date       | content         |
+      | Draft At Root | 2009-03-27 | Random Content. |
+    And I have a "_config.yml" file with content:
+    """
+    collections:
+      puppies:
+        output: true
+
+    collections_dir: collections
+    """
+    When I run jekyll build --drafts
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And the "_site/puppies/rover.html" file should exist
+    And the "_site/2009/03/27/draft-at-root.html" file should not exist
+
+  Scenario: A complex site with collections posts and drafts at various locations
+    Given I have a gathering/_puppies directory
+    And I have a gathering/_posts directory
+    And I have a gathering/_drafts directory
+    And I have a _puppies directory
+    And I have a _posts directory
+    And I have a _drafts directory
+    And I have the following document under the "puppies" collection within the "gathering" directory:
+      | title               | date       | content            |
+      | Rover in Gathering  | 2007-12-31 | content for Rover. |
+    And I have the following document under the puppies collection:
+      | title               | date       | content            |
+      | Rover At Root       | 2007-12-31 | content for Rover. |
+    And I have the following post within the "gathering" directory:
+      | title               | date       | content            |
+      | Post in Gathering   | 2009-03-27 | Totally nothing.   |
+    And I have the following post:
+      | title               | date       | content            |
+      | Post At Root        | 2009-03-27 | Totally nothing.   |
+    And I have the following draft within the "gathering" directory:
+      | title               | date       | content            |
+      | Draft In Gathering  | 2009-03-27 | This is a draft.   |
+    And I have the following draft:
+      | title               | date       | content            |
+      | Draft At Root       | 2009-03-27 | This is a draft.   |
+    And I have a "_config.yml" file with content:
+    """
+    collections:
+      puppies:
+        output: true
+
+    collections_dir: gathering
+    """
+    And I have a "gathering/_puppies/static_file.txt" file that contains "Static content."
+    And I have a gathering/_puppies/nested directory
+    And I have a "gathering/_puppies/nested/static_file.txt" file that contains "Nested Static content."
+    When I run jekyll build --drafts
+    Then I should get a zero exit status
+    And the _site directory should exist
+    And the "_site/puppies/rover-in-gathering.html" file should exist
+    And the "_site/2009/03/27/post-in-gathering.html" file should exist
+    And the "_site/2009/03/27/draft-in-gathering.html" file should exist
+    And the "_site/2009/03/27/draft-at-root.html" file should not exist
+    And the "_site/puppies/rover-at-root.html" file should not exist
+    And I should see exactly "Static content." in "_site/puppies/static_file.txt"
+    And I should see exactly "Nested Static content." in "_site/puppies/nested/static_file.txt"
+    And the _site/gathering directory should not exist
+    And the _site/_posts directory should not exist

features/step_definitions.rb

@@ -98,6 +98,20 @@ end
 
 #
 
+Given(%r!^I have the following (draft|post)s? within the "(.*)" directory:$!) do |type, folder, table|
+  table.hashes.each do |input_hash|
+    title = slug(input_hash["title"])
+    parsed_date = Time.xmlschema(input_hash["date"]) rescue Time.parse(input_hash["date"])
+
+    filename = type == "draft" ? "#{title}.markdown" : "#{parsed_date.strftime("%Y-%m-%d")}-#{title}.markdown"
+
+    path = File.join(folder, "_#{type}s", filename)
+    File.write(path, file_content_from_hash(input_hash))
+  end
+end
+
+#
+
 Given(%r!^I have the following documents? under the (.*) collection:$!) do |folder, table|
   table.hashes.each do |input_hash|
     title = slug(input_hash["title"])
@@ -111,6 +125,16 @@ end
 
 #
 
+Given(%r!^I have the following documents? under the "(.*)" collection within the "(.*)" directory:$!) do |label, dir, table|
+  table.hashes.each do |input_hash|
+    title = slug(input_hash["title"])
+    path = File.join(dir, "_#{label}", "#{title}.md")
+    File.write(path, file_content_from_hash(input_hash))
+  end
+end
+
+#
+
 Given(%r!^I have a configuration file with "(.*)" set to "(.*)"$!) do |key, value|
   config = \
     if source_dir.join("_config.yml").exist?

lib/jekyll/collection.rb

@@ -95,14 +95,13 @@ module Jekyll
         end
     end
 
-    # The directory for this Collection, relative to the site source.
+    # The directory for this Collection, relative to the site source or the directory
+    # containing the collection.
     #
     # Returns a String containing the directory name where the collection
     #   is stored on the filesystem.
     def relative_directory
-      @relative_directory ||= Pathname.new(directory).relative_path_from(
+      @relative_directory ||= "_#{label}"
-        Pathname.new(site.source)
-      ).to_s
     end
 
     # The full path to the directory containing the collection.
@@ -111,7 +110,7 @@ module Jekyll
     #   is stored on the filesystem.
     def directory
       @directory ||= site.in_source_dir(
-        File.join(site.config["collections_dir"], "_#{label}")
+        File.join(container, relative_directory)
       )
     end
 
@@ -125,7 +124,7 @@ module Jekyll
     #   is stored on the filesystem.
     def collection_dir(*files)
       return directory if files.empty?
-      site.in_source_dir(relative_directory, *files)
+      site.in_source_dir(container, relative_directory, *files)
     end
 
     # Checks whether the directory "exists" for this collection.
@@ -204,6 +203,12 @@ module Jekyll
 
     private
 
+    def container
+      @container ||= site.config["collections_dir"]
+    end
+
+    private
+
     def read_document(full_path)
       doc = Jekyll::Document.new(full_path, :site => site, :collection => self)
       doc.read

lib/jekyll/document.rb

@@ -78,12 +78,13 @@ module Jekyll
         collection.label == "posts"
     end
 
-    # The path to the document, relative to the site source.
+    # The path to the document, relative to the collections_dir.
     #
-    # Returns a String path which represents the relative path
+    # Returns a String path which represents the relative path from the collections_dir
-    #   from the site source to this document
+    # to this document.
     def relative_path
-      @relative_path ||= Pathutil.new(path).relative_path_from(site.source).to_s
+      @relative_path ||=
+        Pathutil.new(path).relative_path_from(site.collections_path).to_s
     end
 
     # The output extension of the document.
@@ -400,9 +401,7 @@ module Jekyll
       merge_data!({
         "categories" => (
           Array(data["categories"]) + Utils.pluralized_array_from_hash(
-          data,
+            data, "category", "categories"
-          "category",
-          "categories"
           )
         ).map(&:to_s).flatten.uniq,
       })

lib/jekyll/reader.rb

@@ -62,6 +62,7 @@ module Jekyll
     #
     # Returns nothing.
     def retrieve_posts(dir)
+      return if outside_configured_directory?(dir)
       site.posts.docs.concat(PostReader.new(site).read_posts(dir))
       site.posts.docs.concat(PostReader.new(site).read_drafts(dir)) if site.show_drafts
     end
@@ -130,5 +131,20 @@ module Jekyll
       entries = Dir.chdir(base) { filter_entries(Dir["**/*"], base) }
       entries.delete_if { |e| File.directory?(site.in_source_dir(base, e)) }
     end
+
+    private
+
+    # Internal
+    #
+    # Determine if the directory is supposed to contain posts and drafts.
+    # If the user has defined a custom collections_dir, then attempt to read
+    # posts and drafts only from within that directory.
+    #
+    # Returns true if a custom collections_dir has been set but current directory lies
+    # outside that directory.
+    def outside_configured_directory?(dir)
+      collections_dir = site.config["collections_dir"]
+      !collections_dir.empty? && !dir.start_with?("/#{collections_dir}")
+    end
   end
 end

lib/jekyll/site.rb

@@ -391,6 +391,15 @@ module Jekyll
       end
     end
 
+    # Public: The full path to the directory that houses all the collections registered
+    # with the current site.
+    #
+    # Returns the source directory or the absolute path to the custom collections_dir
+    def collections_path
+      dir_str = config["collections_dir"]
+      @collections_path ||= dir_str.empty? ? source : in_source_dir(dir_str)
+    end
+
     # Limits the current posts; removes the posts which exceed the limit_posts
     #
     # Returns nothing

lib/jekyll/static_file.rb

@@ -40,8 +40,13 @@ module Jekyll
 
     # Returns source file path.
     def path
+      # Static file is from a collection inside custom collections directory
+      if !@collection.nil? && !@site.config["collections_dir"].empty?
+        File.join(*[@base, @site.config["collections_dir"], @dir, @name].compact)
+      else
         File.join(*[@base, @dir, @name].compact)
       end
+    end
 
     # Obtain destination path.
     #