* Switch from nomnom to commander
* Fix#1489
* --macro repeatable option replaced with single --macros option with ";"
separator (until https://github.com/tj/commander.js/issues/674)
* Add missing --max-expand option
* Replace expired --unicode-text-in-math-mode option with --strict
(https://github.com/Khan/KaTeX/issues/1483#issuecomment-405108291)
* Add instructions for how to run in the opening comments
* Clean up option doc strings
* Implement @ylemkimon's comments
* Fix try/const bug
* Fix use of --selenium-ip
It was a quirk that --seleniumIP worked. --help lists --selenium-ip.
* Implement comments, restore --macro repeatable argument
* Cleanup
* Restore 0 default for maxExpand
* Fix more comments from @ylemkimon
* npm run dist -> npm run build
In preparation for #1500
* Fix line length error
* Add regenerate option to the screenshotter
Save new screenshot when match fails
* Ignore Chrome debug log file
* CircleCI: Generate only failed screenshots
* CircleCI Test
* Revert "CircleCI Test"
This reverts commit 5d3afb2602f32470eeba7767748faba177ba933e.
* Rename `regenerate` to `new`
* Add ` /test/screenshotter/new/` to .gitignore
* Update texcmp
I found that the existing docker rules for texcmp did not make a working
installation of LaTeX; in particular, it was missing a lot of the recently
added packages. I updated to Ubuntu 17.10 and tweaked the package listings
until things compiled again. Now I can run texcmp again.
* texlive-font-utils not necessary
* Add CircleCI config for online screenshot verification/generation
* Remove `Checkout unicode fonts` step
Unicode-fonts are now a submodule, which is checked out in `Checkout submodule` step.
* Always generate new screenshots
* Group more test npm scripts into `test:`
* Update documentations on the development workflow
* Update .travis.yml to match updated NPM scripts
* Run npm install in Travis
* Create a separate entry point for webpack
Created a webpack entry point for KaTeX, which imports katex.less. As
flow[1] and jest[2] doesn't support CSS modules natively, a separate
entry point is used and it is not flowtyped.
[1] https://gist.github.com/lambdahands/d19e0da96285b749f0ef
[2] https://facebook.github.io/jest/docs/en/webpack.html
* Use webpack to build files
* Made webpack.config.js export valid webpack configuration
* Use:
browserify -> webpack
babelify -> babel-loader
UglifyJS CLI -> UglifyJsPlugin
Less CLI -> less-loader
cleancss -> cssnano in css-loader
build/fonts -> file-loader
* Inline CSS(Less) using style-loader and export them using
ExtractTextPlugin
* Add `watch` npm script calling `webpack --watch`
* Improve local testing(webpack-dev-server)
* Made webpackDevServer export a valid webpack configuration
* Compile Less and inline CSS using less-loader and style-loader
* Instead of copying files serve files from /static and use file-loader
* Remove old server.js and its dependencies
* Use webpack-dev-server in Screenshotter
* Include contrib in webpack-dev-server
+ Moved common configurations to webpack.common.js
* Rename webpackDevServer.js to webpack.dev...
to be consistent, avoid confusion with webpack-dev-server and follow
webpack configuration naming convention.
* Remove unnecessary conditional output.path
* Use map instead of reduce
+ Add comments regarding function arguments
* Remove unnecessary mkdir and clean build/* before build
* Use katex as external dependency instead of global variable in contrib
Fixes#692.
* Unblock codes as they are built as a module
* Update package-lock.json
* Add comments regarding devServer option
* Lint renamed webpack.dev.js
a0d8b33
* Export ES6 module and expose its default export
* Revert "Browserify hotfix (#1057)"
This reverts commit f6b509123b.
* Enables colors on the console when running the dev server in Screenshotter
* Add context to webpack configuration
Allows webpack to be run from other directories
* Move `rm -rf build/*` to npm scripts
* Check dependencies before build
* Move UglifyJsPlugin into config creation
* Let webpack handle ES6 modules
Do not transform modules to commonjs in Babel. However Jest doesn't not
support ES6 modules, so transfrom modules to commonjs when NODE_ENV is
`test`.
* Add documentation on testing in IE 9 and 10 using webpack-dev-server
Changed version range to include IE-compatible version
* Modernize font creation
* Automate the build process, instead of writing a complicated manual
* Update sources used by the docker image away from google code
* Exclude MathJax-dev checkout from docker image, to allow image reuse
* Try to keep the individual image layers small by combining RUN commands
and removing temporary files used only during each individual step
* Allow creating fonts from local directory
Also use hash of Dockerfile to compute image tag, in order to ensure
creation of a new image if the Dockerfile changes.
* make glob syntax work on MacOS
* Make KaTeX work in Quirks mode
Summary: In issue #601, it was noticed that the KaTeX bug with the fraction
bars overlapping the text was occuring with an XHTML doctype. This indicated
that the bug we were seeing was caused by both quirks mode and limited-quirks
mode (which is a version of quirks mode with fewer quirks and is enabled for
various doctypes including some XHTML ones).
Based on the [quirks spec](https://quirks.spec.whatwg.org/), it appears that
there are only two quirks in limited-quirks mode, both having to do with a
line-height calculation. @gagern figured out that if we added some zero-width
spaces in our elements, we would stop triggering the quirk, which would make
our fractions render correctly in limited-quirks mode.
I implemented that change, and ran the screenshotter in limited-quirks mode.
There were several other places that suffered from the same quirk, but were
also easily fixed via adding zero-width spaces. I then ran the screenshotter in
quirks mode, and discovered that (once an appropriate meta charset was added),
everything looked correct still.
So, this diff fixes all of the places that the limited-quirks mode quirks
affect our rendering, and removes the warning about rendering in quirks mode.
I also added support to our screenshotter to render things in both no-quirks
and quirks mode, to ensure that things don't break in the future. I copied the
non-quirks images to the quirks images, and ran the screenshotter with
`--verify` to make sure that they look the same.
I have some thoughts that I'd like to hear opinions about:
- I'm not super happy with how the screenshot tests work. Ideally we'd test
both quirks mode and non-quirks mode against the same images, since we'd
like them to be the same. I'm not sure how to make that work well, though,
since then people wouldn't be able to tell if it's a quirks-mode problem or
not.
- I removed the doctype in the testing page file, so all testing would now be
done in quirks-mode. Not sure if we really want that.
- I need to test this in IE, but it looks like the trailing commas change we
made with eslinting is causing problems (cause IE doesn't like trailing
commas).
Test Plan:
- `./dockers/Screenshotter/screenshotter.sh --verify`
* Compare quirks mode against same screenshot files
Now the screenshotter itself can run more than one mode. It does serve the
HTML file from its own JavaScript code now, so that it can include different
doctype headers without needing distinct files for each. There is a
provision to mark specific tests as quirky in case they produce different
results depending on the mode.
* Some cleaning up and comments
* Restore access to the babelified version of the HTML page for screenshots
* Reference unicode fonts using absolute path names
This avoids issues caused by the fact that the dynamically generated
ss-render.html is mounted to a different location than the test.html from
which it is derived.
* do chrome screenshots first
* remove commented out code, simplify hadle_search_string call
Previously, we added a `oneDone` promise *every time a test attempt
failed*. This decremented the count of remaining tests once per
failed attempt. As a result, Chrome tests didn't run to the end.
The slowest part of screenshotter tests is the page load, probably
because so many assets must be loaded over the slow docker
connection. On my laptop this takes ~4s per test. The new default
avoids this cost by rendering new TeX on the existing
page. For second and subsequent tests, use `executeAsyncScript`
to call KaTeX, rather than performing a full page + asset load.
(If too many errors happen in `--verify` mode, we fall back to
full loads.) The `--reload` flag will enable the previous behavior.
On my laptop, a full verify (chrome + FF) used to take 12m20s+.
It now takes 2m23s, a speed up of 6x.
* Add babel transform-class-properties to have static class properties
* Upgrade Lexer and Parser files to use ES6 classes
* Update eslint max line length to 90 character (more indent because of using ES6 classes)
* Upgrade eslint and jasmin to support ES stage-2 features
* Use static properties to place constants near their functions
* Migrate all remaining sources to ES6 syntax
* Increase eslint max line length to 84
* Remove non-babelified endpoint in dev server.js
* Clean up server.js functions after removing browserified
* Make screenshotter not to use babel endpoint as we babelify everything now
* Update texcmp to ubuntu 17.04 and avoid mounted host directory
Switch to linux 17.04 in order to have a version of nodejs which understands
ES6 syntax, for the sake of a consistent codebase.
Avoid mounting the host directory, but use “docker cp” instead to transfer
files between host and container. This should avoid ownership and
permission issues.
Support macros with positional arguments.
Fix one overline example which caused LaTeX failure due to missing braces.
* Extract texcmp results as current user
This allows running the texcmp.sh script using sudo on afs.
* Change texcmp conversion to gray
As per #708, this should increase compatibility with older versions of
imagemagick, and might also do a better job of preserving the original sRGB
color space.
* Abandon tar and use plain docker cp instead
Thanks to Erik Demaine for suggesting this.
* Move npm install into creation of texcmp docker image
As babelify is slow, it may be desriable to not run it during development.
This is OK if the browser is recent enough to understand ES6 natively.
(This does not include current Firefox due to it having problems with
for(const … in …), https://bugzilla.mozilla.org/show_bug.cgi?id=1094995.)
For older browsers, or to check issues possibly introduced by babelify,
adding /babel as the first component of the path will switch to a version
which has been processed by babelify. This is also used for screenshots.
This reverts commit 4d2e46e7f6.
Having trailing commans makes diffs easier to read as it avoids modifying a
line just to add a trailing comma if there is another item to add at the end
of a list. There are plans to switch to ES6 notation and to translate that
to ES5 as part of the build process. Since that translation would remove
trailing commas, the IE9 problems that originally motivated the commit
should vanish soon.
All these version ranges include the latest version at the time of this
commit, except for the selenium webdriver. There version 3 is incompatible
with version 2, and switching to a version 3 library appears to require
switching to version 3 docker images as well, which would entail using
different browser versions which in turn would lead to differences for a
large number of screenshots. That doesn't appear warranted at this time.
Summary: IE 9 doesn't like trailing commas. When we introduced eslint, we added
a bunch of trailing commas, which makes IE 9 sad.
Test Plan:
- `make lint`
- Visit http://localhost:7936/ using IE 9 on browserstack.
- See that the math loads, and there are no errors in the F12 developer tools.
@kevinb
Experimenting with selenium-webdriver@3.0.1 and Firefox 50.1.0 I observed
screenshots having a height of merely 8 pixels. Presumably the margin or
padding of an otherwise empty document. So in order to get the actual size
of the document area, the screenshotter now loads a document which fills the
entire viewport.
* Exit cleanly after invalid screenshot test name
This avoids waiting forever.
* Auto-detect host IP in Docker for Mac environment
This checks all available network addresses to find one which the Selenium
container can connect to. That way we don't have to analyze network
settings or similar to figure out the main public IP address of the machine.
* Make server less sensitive to current working directory
That way it becomes possible to run screenshotter.js from within the
Screenshotter directory, and still server all files as intended.
This ensures that running screenshots as indicated by the screenshotter
readme will correctly render the Unicode examples, even if not run on
Travis. It also fixes the commit ID of the unicode fonts.
One alternative would have been using a git submodule. But as many tools
will initialize submodules automatically, that would mean big downloads for
anyone using such a tool to clone KaTeX.
Another alternative would be tweaking the server to fetch the files on
demand if the local copy is unavailable. But that would cause additional
network overhead, so grabbing the files just once should be simpler.
* Ensure bit depth 8
* Print affected file if PNG failed to read (e.g. due to wrong bit depth)
* Disable running Kern test case through TeX as doing so fails
Since boot2docker has been superseded by docker-machine, we need this to
support developers on OS X. The changes to the bash script ensure that we
clean up our containers even if taking screenshots gets hung and requires a
keyboard interrupt, as happens if the IP addresses were guessed incorrectly.
Sometimes a page might not be ready when the screenshot is taken, due to
fonts still being loaded or some such. In --verify mode this is taken care
of by repeated attempts. But when creating a screenshot for the first time,
it might be desirable to wait a given number of seconds for things to settle
down. So this commit introduces a --wait option to the screenshotter.
Summary
We'd like contributors to use the same linter and lint rules that we use
internally. This diff swaps out eslint for jshint and fixes all lint failures
except for the max-len failures in the test suites.
Test Plan:
- ka-lint src
- make lint
- make test
Reviewers: emily
Otherwise it could happen that some Mozilla page gets shown which has a
minimal size larger than the 786px we're requesting. And the screenshot
will span that entire page even if the window is smaller, resulting in a
failure to adjust screenshot size.
See http://kb.mozillazine.org/Browser.startup.homepage_override.mstone
and http://kb.mozillazine.org/Browser.startup.page for details.
Just in case, we also include the docker image digests in the travis build
log, to increase chances of reproducing what we get there.
Now Travis can run the screenshotter in verification mode. The files in the
repository will be seen as the expected outcome, and if the actual result
differs from that, it might be attempted four more times before the test
case is actually deemed failed. A timeout between page load and screenshot
should allow any possible font issues to settle down.
The combination of jspngopt and pako should eliminate possible causes for
different PNG encodings, although the core reason for #325 remains unknown.
Pako has poorer compression rates than native libz, but optimization can
counter that effect, and actually reduce the size of the screenshots.
The screenshots for LimitControls and UnsupportedCmds on Firefox used to
exhibit subpixel rendering before, for reasons unknown. The regenerated
versions don't exhibit this. See #324 for a discussion.
Summary:
Update the MathJaxFonts `Dockerfile` to enable the use of
`ttfautohint` which hints our fonts better. Also, fix the location that
batik is downloaded from, update some formatting in the `Dockerfile`,
and update the fonts and metrics.
Test Plan:
- Compare a rendered `= - A z 4 \Longrightarrow \Sigma \Biggl(` in
Chrome on Windows at font sizes 10px to 20px before and after this
change.
- See that characters look about the same, or better.
- See that the screenshot tests didn't change in firefox (maybe firefox
runs the same autohinting algorithm that we do?), and don't visually
change in chrome
Reviewers: kevinb, alpert
Reviewed By: kevinb, alpert
Differential Revision: https://phabricator.khanacademy.org/D18977
Escaping TeX in JSON as query strings is a pain: you have to double all the
\\, you have to escape the & and the #, you can't easily include line breaks
for readability, and so on. YAML solves most of these problems for most of
the situations. Now each test case can be structured, while simple test
cases only consist of a line of verbatim TeX code, with no escaping.
The most troublesome items remaining are lines starting in { since in YAML
these would denote inline mapping types. We use block notation for these.
This avoids one of the few requirements we have left: you no longer have to
start a KaTeX development server, the script will do it for you, using a
random port number.
To reproduce the old behaviour, explicitely state --katex-port=7936.
Since the Selenium images are available for download, and downloading them
is usually faster than building them from scratch, this makes taking
screenshots easier. Furthermore, since the Selenium image is not specific
to KaTeX, it could as well be used for other purposes, thus saving space
since a single image can be used in multiple projects.
This change also deals with the non-determinism in the Lap screenshot:
We detect the one known (and accepted) alternate rendering and change the
output file name to Lap_alt in this case. So either Lap or Lap_alt gets
saved to, and if the image is different from both, then one of these files
will show a modification. On the other hand, if it is equal to either of
these, then the matching one will get overwritten, showing no change.
The same test cases we use for our screenshots from Firefox are now also
being rendered by pdflatex, so the resulting images can be used as reference
for how things are supposed to look (if we concentrate on compatibility with
LaTeX). To make comparisons even easier, the differences between LaTeX and
Firefox snapshots are rendered in a visual way, using different colors.
Discussed in pull request #268.
Summary:
Create our own screenshotting script which takes screenshots. This
improves over huxley for a couple reasons:
- It makes the screenshots the correct size (for some reason, huxley struggles
with this).
- Its configuration matches more with what we want (we don't need multiple
screenshots or interaction, we just want a single static shot)
- It runs faster
I also changed the docs to reflect this change.
Test Plan:
- Make sure all of the tests that were in the Huxleyfile are now in ss_data.json
- Run the screenshotter docker
- Make sure all of the images look reasonable and don't change (except
sometimes the Lap test, which has some strange pixel-positioning
differences...)
Reviewers: kevinb
Reviewed By: kevinb
Differential Revision: https://phabricator.khanacademy.org/D16731
Summary:
To complement pull request #142, add the ability to automatically
generate the woff2 files from within the docker. Add a script for copying out
the generated font files to make things easier. Remove the KaTeX_Greek font
files as well as the KaTeX_Win ones, since we don't use them and they're oddly
non-woff and non-eot specific?
Test Plan:
- Build a new docker from the new Dockerfile
- Make sure the woff2_compress program compiles
- Follow the instructions in the README to build the fonts
- Make sure the fonts correctly build
- Make sure the copy_fonts.sh script copies the fonts out
Reviewers: alpert
Reviewed By: alpert
Differential Revision: http://phabricator.khanacademy.org/D13637
Update huxley, and then use the Dockerfile to create new huxley screenshots.
Test plan:
- Make sure generating new huxley screenshots works
- Ensure that re-generating huxley screenshots produces no changes
Auditors: alpert
Summary:
Add the dockerfile that I've been using to make huxley screenshots so
others can test their changes by making huxley screenshots. Include a readme
about how to use it correctly.
Test Plan: - Use the dockerfile by running the given commands, see it works
Reviewers: alpert
Reviewed By: alpert
Differential Revision: http://phabricator.khanacademy.org/D12773
The OS/2 values in the generated fonts were incorrect, so this corrects them. To
do this, I added some files to the khan/MathJax-dev repo, and updated the
dockerfile to use the new repo.
Test Plan:
- View the test page with chrome and firefox on linux, chrome and safari on mac,
and IE8 and IE9 on windows, and see that none of the characters are clipped.
Auditors: alpert
Summary:
Also, rename all of our uses of fonts to use the uppercased versions. We want to
use the uppercase versions because it makes updating and modifying the fonts
much easier (since the font names inside the actual font files are uppercased).
Test Plan:
- Make sure the huxley screenshots look good (You can compare a diff of them
on github at
f90d093361
By my eye, it seems like some things have moved up ~1/2 pixel, and some of
the fonts have maybe slightly changed shape, like the large `b` in
SizingBaseline)
Reviewers: alpert
Reviewed By: alpert
Differential Revision: http://phabricator.khanacademy.org/D11979
