dan
/
jekyll
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Drop support for pygments as syntax-highlighter (#7118) 

Merge pull request 7118
This commit is contained in:
Ashwin Maroli 2018-09-27 17:59:46 +05:30 committed by jekyllbot
parent deff194cbf
commit 4707017936
6 changed files with 20 additions and 233 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Gemfile
View File

@ -75,7 +75,6 @@ group :jekyll_optional_dependencies do
platform :ruby, :mswin, :mingw, :x64_mingw do platform :ruby, :mswin, :mingw, :x64_mingw do
gem "classifier-reborn", "~> 2.2.0" gem "classifier-reborn", "~> 2.2.0"
gem "liquid-c", "~> 3.0" gem "liquid-c", "~> 3.0"
gem "pygments.rb", "~> 1.0"
gem "yajl-ruby", "~> 1.4" gem "yajl-ruby", "~> 1.4"
end end

19
docs/_docs/liquid/tags.md
View File

@ -14,16 +14,23 @@ If you have page snippets that you use repeatedly across your site, an
## Code snippet highlighting ## Code snippet highlighting
Jekyll has built in support for syntax highlighting of over 60 languages Jekyll has built in support for syntax highlighting of over 100 languages
thanks to [Rouge](http://rouge.jneen.net). Rouge is the default highlighter thanks to [Rouge](http://rouge.jneen.net). Rouge is the default highlighter
in Jekyll 3 and above. To use it in Jekyll 2, set `highlighter` to `rouge` in Jekyll 3 and above. To use it in Jekyll 2, set `highlighter` to `rouge`
and ensure the `rouge` gem is installed properly. and ensure the `rouge` gem is installed properly.
Alternatively, you can use [Pygments](http://pygments.org) to highlight Alternatively, you can use [Pygments](http://pygments.org) to highlight your
your code snippets. To use Pygments, you must have Python installed on your code snippets in Jekyll 3.x and below. To use Pygments, you must have Python
system, have the `pygments.rb` gem installed and set `highlighter` to installed on your system, have the `pygments.rb` gem installed and set
`pygments` in your site's configuration file. Pygments supports [over 100 `highlighter` to `pygments` in your site's configuration file. Pygments
languages](http://pygments.org/languages/) supports [over 100 languages](http://pygments.org/languages/)
<div class="note info">
<p>Using Pygments has been deprecated and will not be officially supported in
Jekyll 4, meaning that the configuration setting <code>highlighter: pygments</code>
will automatically fall back to using <em>Rouge</em> which is written in Ruby
and 100% compatible with stylesheets for Pygments.</p>
</div>
To render a code block with syntax highlighting, surround your code as follows: To render a code block with syntax highlighting, surround your code as follows:

2
docs/_tutorials/orderofinterpretation.md
View File

@ -89,7 +89,7 @@ console.log('alert');
``` ```
{% endraw %} {% endraw %}
The `highlight` tag *is* Liquid. (Liquid passes the content to Rouge or Pygments for syntax highlighting.) As a result, this code will actually convert to HTML with syntax highlighting. Jekyll does not need the Markdown filter to process `highlight` tags. The `highlight` tag *is* Liquid. (Liquid passes the content to Rouge for syntax highlighting.) As a result, this code will actually convert to HTML with syntax highlighting. Jekyll does not need the Markdown filter to process `highlight` tags.
### Liquid mixed with JavaScript isn't rendered ### Liquid mixed with JavaScript isn't rendered

8
features/site_configuration.feature
View File

@ -73,14 +73,6 @@ Feature: Site configuration
And the _site directory should exist And the _site directory should exist
And I should see "<a href=\"https://www.google.com\">Google</a>" in "_site/index.html" And I should see "<a href=\"https://www.google.com\">Google</a>" in "_site/index.html"
Scenario: Highlight code with pygments
Given I have an "index.html" page that contains "{% highlight ruby %} puts 'Hello world!' {% endhighlight %}"
When I run jekyll build
Then I should get a zero exit status
And the _site directory should exist
And I should see "Hello world!" in "_site/index.html"
And I should see "class=\"highlight\"" in "_site/index.html"
Scenario: Highlight code with rouge Scenario: Highlight code with rouge
Given I have an "index.html" page that contains "{% highlight ruby %} puts 'Hello world!' {% endhighlight %}" Given I have an "index.html" page that contains "{% highlight ruby %} puts 'Hello world!' {% endhighlight %}"
And I have a configuration file with "highlighter" set to "rouge" And I have a configuration file with "highlighter" set to "rouge"

47
lib/jekyll/tags/highlight.rb
View File

@ -33,14 +33,12 @@ module Jekyll
suffix = context["highlighter_suffix"] || "" suffix = context["highlighter_suffix"] || ""
code = super.to_s.gsub(%r!\A(\n|\r)+|(\n|\r)+\z!, "") code = super.to_s.gsub(%r!\A(\n|\r)+|(\n|\r)+\z!, "")
is_safe = !!context.registers[:site].safe
output = output =
case context.registers[:site].highlighter case context.registers[:site].highlighter
when "pygments"
render_pygments(code, is_safe)
when "rouge" when "rouge"
render_rouge(code) render_rouge(code)
when "pygments"
render_pygments(code, context)
else else
render_codehighlighter(code) render_codehighlighter(code)
end end
@ -49,20 +47,6 @@ module Jekyll
prefix + rendered_output + suffix prefix + rendered_output + suffix
end end
def sanitized_opts(opts, is_safe)
if is_safe
Hash[[
[:startinline, opts.fetch(:startinline, nil)],
[:hl_lines, opts.fetch(:hl_lines, nil)],
[:linenos, opts.fetch(:linenos, nil)],
[:encoding, opts.fetch(:encoding, "utf-8")],
[:cssclass, opts.fetch(:cssclass, nil)],
].reject { |f| f.last.nil? }]
else
opts
end
end
private private
OPTIONS_REGEX = %r!(?:\w="[^"]*"|\w=\w|\w)+! OPTIONS_REGEX = %r!(?:\w="[^"]*"|\w=\w|\w)+!
@ -86,29 +70,10 @@ module Jekyll
options options
end end
def render_pygments(code, is_safe) def render_pygments(code, _context)
Jekyll::External.require_with_graceful_fail("pygments") unless defined?(Pygments) Jekyll.logger.warn "Warning:", "Highlight Tag no longer supports rendering with Pygments."
Jekyll.logger.warn "", "Using the default highlighter, Rouge, instead."
highlighted_code = Pygments.highlight( render_rouge(code)
code,
:lexer => @lang,
:options => sanitized_opts(@highlight_options, is_safe)
)
if highlighted_code.nil?
Jekyll.logger.error <<~MSG
There was an error highlighting your code:
#{code}
While attempting to convert the above code, Pygments.rb returned an unacceptable value.
This is usually a timeout problem solved by running `jekyll build` again.
MSG
raise ArgumentError, "Pygments.rb returned an unacceptable value "\
"when attempting to highlight some code."
end
highlighted_code.sub('<div class="highlight"><pre>', "").sub("</pre></div>", "")
end end
def render_rouge(code) def render_rouge(code)

176
test/test_tags.rb
View File

@ -134,182 +134,6 @@ class TestTags < JekyllUnitTest
end end
end end
context "in safe mode" do
setup do
@tag = highlight_block_with_opts("text ")
end
should "allow linenos" do
sanitized = @tag.sanitized_opts({ :linenos => true }, true)
assert_equal true, sanitized[:linenos]
end
should "allow hl_lines" do
sanitized = @tag.sanitized_opts({ :hl_lines => %w(1 2 3 4) }, true)
assert_equal %w(1 2 3 4), sanitized[:hl_lines]
end
should "allow cssclass" do
sanitized = @tag.sanitized_opts({ :cssclass => "ahoy" }, true)
assert_equal "ahoy", sanitized[:cssclass]
end
should "allow startinline" do
sanitized = @tag.sanitized_opts({ :startinline => true }, true)
assert_equal true, sanitized[:startinline]
end
should "strip unknown options" do
sanitized = @tag.sanitized_opts({ :light => true }, true)
assert_nil sanitized[:light]
end
end
context "with the pygments highlighter" do
setup do
if jruby?
then skip(
"JRuby does not support Pygments."
)
end
end
context "post content has highlight tag" do
setup do
fill_post("test", "highlighter" => "pygments")
end
should "not cause a markdown error" do
refute_match(%r!markdown\-html\-error!, @result)
end
should "render markdown with pygments" do
assert_match(
%(<pre><code class="language-text" data-lang="text">) +
%(<span></span>test</code></pre>),
@result
)
end
should "render markdown with pygments with line numbers" do
assert_match(
%(<pre><code class="language-text" data-lang="text">) +
%(<span></span><span class="lineno">1 </span>test</code></pre>),
@result
)
end
end
context "post content has highlight with file reference" do
setup do
fill_post("./jekyll.gemspec", "highlighter" => "pygments")
end
should "not embed the file" do
assert_match(
%(<pre><code class="language-text" data-lang="text"><span></span>) +
%(./jekyll.gemspec</code></pre>),
@result
)
end
end
context "post content has highlight tag with UTF character" do
setup do
fill_post("Æ", "highlighter" => "pygments")
end
should "render markdown with pygments line handling" do
assert_match(
%(<pre><code class="language-text" data-lang="text">) +
%(<span></span>Æ</code></pre>),
@result
)
end
end
context "post content has highlight tag with preceding spaces & lines" do
setup do
code = <<~EOS
[,1] [,2]
[1,] FALSE TRUE
[2,] FALSE TRUE
EOS
fill_post(code, "highlighter" => "pygments")
end
should "only strip the preceding newlines" do
assert_match(
%(<pre><code class=\"language-text\" data-lang=\"text\">) +
%(<span></span> [,1] [,2]),
@result
)
end
end
context "post content has highlight tag " \
"with preceding spaces & lines in several places" do
setup do
code = <<~EOS
[,1] [,2]
[1,] FALSE TRUE
[2,] FALSE TRUE
EOS
fill_post(code, "highlighter" => "pygments")
end
should "only strip the newlines which precede and succeed the entire block" do
assert_match(
%(<pre><code class=\"language-text\" data-lang=\"text\"><span></span>) +
%( [,1] [,2]\n\n\n[1,] FALSE TRUE\n[2,] FALSE TRUE</code></pre>),
@result
)
end
end
context "post content has highlight tag with " \
"preceding spaces & Windows-style newlines" do
setup do
fill_post "\r\n\r\n\r\n [,1] [,2]", "highlighter" => "pygments"
end
should "only strip the preceding newlines" do
assert_match(
%(<pre><code class="language-text" data-lang="text"><span></span>) +
%( [,1] [,2]),
@result
)
end
end
context "post content has highlight tag with only preceding spaces" do
setup do
code = <<~EOS
[,1] [,2]
[1,] FALSE TRUE
[2,] FALSE TRUE
EOS
fill_post(code, "highlighter" => "pygments")
end
should "only strip the preceding newlines" do
assert_match(
%(<pre><code class=\"language-text\" data-lang=\"text\"><span></span>) +
%( [,1] [,2]),
@result
)
end
end
end
context "with the rouge highlighter" do context "with the rouge highlighter" do
context "post content has highlight tag" do context "post content has highlight tag" do
setup do setup do