Merge branch texjs:master into master

.github/dependabot.yml

+version: 2
+updates:
+  - package-ecosystem: "npm"
+    directory: "/"
+    schedule:
+      interval: "daily"
+
+    pull-request-branch-name:
+      # Separate sections of the branch name with a hyphen
+      # Docker images use the branch name and do not support slashes in tags
+      # https://github.com/overleaf/google-ops/issues/822
+      # https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates#pull-request-branch-nameseparator
+      separator: "-"
+
+    # Block informal upgrades -- security upgrades use a separate queue.
+    # https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates#open-pull-requests-limit
+    open-pull-requests-limit: 0

.gitignore

@@ -11,7 +11,6 @@ clsi
 filestore
 track-changes
 docstore
-tags
 chat
 spelling
 real-time
@@ -23,4 +22,4 @@ tmp
 db.sqlite
 
 .DS_Store
-.vagrant
\ No newline at end of file
+.vagrant

CONTRIBUTING.md

@@ -4,7 +4,7 @@ Contributing to ShareLaTeX
 Thank you for reading this! If you'd like to report a bug or join in the development
 of ShareLaTeX, then here are some notes on how to do that.
 
-*Note that ShareLaTeX is actually made up of many seperate repositories (a list is available
+*Note that ShareLaTeX is actually made up of many separate repositories (a list is available
 [here](https://github.com/sharelatex/sharelatex/blob/master/README.md#other-repositories)).* 
 
 Reporting bugs and opening issues

Dockerfile

@@ -58,7 +58,7 @@ RUN cd /var/www/sharelatex \
 RUN cd /var/www/sharelatex \
 &&    bash ./bin/compile-services
 
-# Links CLSI sycntex to its default location
+# Links CLSI synctex to its default location
 # ------------------------------------------
 RUN ln -s /var/www/sharelatex/clsi/bin/synctex /opt/synctex
 
@@ -77,6 +77,7 @@ ADD ${baseDir}/nginx/sharelatex.conf /etc/nginx/sites-enabled/sharelatex.conf
 # Configure log rotation
 # ----------------------
 ADD ${baseDir}/logrotate/sharelatex /etc/logrotate.d/sharelatex
+RUN chmod 644 /etc/logrotate.d/sharelatex
 
 
 # Copy Phusion Image startup scripts to its location
@@ -93,6 +94,8 @@ ENV WEB_API_USER "sharelatex"
 
 ENV SHARELATEX_APP_NAME "Overleaf Community Edition"
 
+ENV OPTIMISE_PDF "true"
+
 
 EXPOSE 80
 

Dockerfile-base

@@ -7,6 +7,11 @@ FROM phusion/baseimage:0.11
 ENV baseDir .
 
 
+# Makes sure LuaTex cache is writable
+# -----------------------------------
+ENV TEXMFVAR=/var/lib/sharelatex/tmp/texmf-var
+
+
 # Install dependencies
 # --------------------
 RUN apt-get update \

Gruntfile.coffee

@@ -193,7 +193,7 @@ module.exports = (grunt) ->
 				grunt.log.errorlns """
 				!!!!!!!!!!!!!! MONGO ERROR !!!!!!!!!!!!!!
 
-				ShareLaTeX can not talk to the mongdb instance
+				ShareLaTeX can not talk to the mongodb instance
 
 				Check the mongodb instance is running and accessible on env var SHARELATEX_MONGO_URL
 				

README.md

@@ -39,9 +39,9 @@ If you are upgrading from a previous version of Overleaf, please see the [Releas
 
 ## Other repositories
 
-This repository does not contain any code. It acts a wrapper and toolkit for managing the many different Overleaf  services. These each run as their own Node.js process and have their own Github repository. These are all downloaded and set up when you run `grunt install`
+This repository does not contain any code. It acts a wrapper and toolkit for managing the many different Overleaf  services. These each run as their own Node.js process and have their own GitHub repository. These are all downloaded and set up when you run `grunt install`
 
-| Service | Description | 
+| Service | Description |
 | ------- | ----------- |
 | **[web](https://github.com/overleaf/web)** | The front facing web server that serves all the HTML pages, CSS and JavaScript to the client. Also contains a lot of logic around creating and editing projects, and account management. |
 | **[document-updater](https://github.com/overleaf/document-updater)** | Processes updates that come in from the editor when users modify documents. Ensures that the updates are applied in the right order, and that only one operation is modifying the document at a time. Also caches the documents in redis for very fast but persistent modifications. |
@@ -51,7 +51,6 @@ This repository does not contain any code. It acts a wrapper and toolkit for man
 | **[filestore](https://github.com/overleaf/filestore)** | An API for performing CRUD (Create, Read, Update and Delete) operations on binary files (like images) stored in Overleaf. |
 | **[track-changes](https://github.com/overleaf/track-changes)** | An API for compressing and storing the updates applied to a document, and then rendering a diff of the changes between any two time points. |
 | **[chat](https://github.com/overleaf/chat)** | The backend API for storing and fetching chat messages. |
-| **[tags](https://github.com/overleaf/tags)** | The backend API for managing project tags (folders). |
 | **[spelling](https://github.com/overleaf/spelling)** | An API for running server-side spelling checking on Overleaf documents. |
 
 ## Overleaf Docker Image

bin/compile-services

@@ -12,9 +12,6 @@ grep 'name:' config/services.js | \
 			web)
 				npm run webpack:production
 				;;
-			real-time)
-				npm run compile:all
-				;;	
 			*)
 				echo "$service doesn't require a compilation"
 				;;

docker-compose.yml

@@ -11,7 +11,6 @@ services:
                 condition: service_healthy
             redis:
                 condition: service_started
-        privileged: true
         ports:
             - 80:80
         links:
@@ -20,7 +19,7 @@ services:
         volumes:
             - ~/sharelatex_data:/var/lib/sharelatex
             ########################################################################
-            ####  Server Pro: Un-comment the following line to mount the docker ####
+            ####  Server Pro: Uncomment the following line to mount the docker  ####
             ####             socket, required for Sibling Containers to work    ####
             ########################################################################
             # - /var/run/docker.sock:/var/run/docker.sock
@@ -39,7 +38,7 @@ services:
 
             # Enables Thumbnail generation using ImageMagick
             ENABLE_CONVERSIONS: 'true'
-            
+
             # Disables email confirmation requirement
             EMAIL_CONFIRMATION_DISABLED: 'true'
 
@@ -70,7 +69,7 @@ services:
             # SHARELATEX_EMAIL_SMTP_PASS:
             # SHARELATEX_EMAIL_SMTP_TLS_REJECT_UNAUTH: true
             # SHARELATEX_EMAIL_SMTP_IGNORE_TLS: false
-            # SHARELATEX_CUSTOM_EMAIL_FOOTER: "<div>This system is run by department x </div>"
+            # SHARELATEX_CUSTOM_EMAIL_FOOTER: "This system is run by department x"
 
             ################
             ## Server Pro ##
@@ -103,7 +102,7 @@ services:
 
     mongo:
         restart: always
-        image: mongo
+        image: mongo:4.0
         container_name: mongo
         expose:
             - 27017
@@ -117,7 +116,7 @@ services:
 
     redis:
         restart: always
-        image: redis
+        image: redis:5
         container_name: redis
         expose:
             - 6379

hotfix/2.3.1/Dockerfile

+FROM sharelatex/sharelatex:2.3.0
+
+
+# Patch: Fixes NPE when invoking synctex (https://github.com/overleaf/overleaf/issues/756)
+ADD check-clsi-setting-exists.patch /var/www/sharelatex/clsi/app/js/check-clsi-setting-exists.patch
+RUN cd /var/www/sharelatex/clsi/app/js && \
+    patch < check-clsi-setting-exists.patch

hotfix/2.3.1/check-clsi-setting-exists.patch

+--- a/app/js/CompileManager.js
++++ b/app/js/CompileManager.js
+@@ -536,7 +536,7 @@ module.exports = CompileManager = {
+       compileName,
+       command,
+       directory,
+-      Settings.clsi != null ? Settings.clsi.docker.image : undefined,
++      Settings.clsi && Settings.clsi.docker ? Settings.clsi.docker.image : undefined,
+       timeout,
+       {},
+       function(error, output) {

hotfix/2.4.1/Dockerfile

+FROM sharelatex/sharelatex:2.4.0
+
+
+# Patch: Fixes missing dependencies on web startup (https://github.com/overleaf/overleaf/issues/767)
+RUN cd /var/www/sharelatex/web && \
+    npm install i18next@^19.6.3 i18next-fs-backend@^1.0.7 i18next-http-middleware@^3.0.2

hotfix/2.4.2/Dockerfile

+FROM sharelatex/sharelatex:2.4.1
+
+
+# Patch: Fixes anonymous read/write sharing
+COPY anonymous-metadata.patch ${baseDir}
+RUN cd ${baseDir} && patch -p0 < anonymous-metadata.patch
+
+# Patch: Fixes left footer with html text
+COPY left-footer-skip-translation.patch ${baseDir}
+RUN cd ${baseDir} && patch -p0 < left-footer-skip-translation.patch

hotfix/2.4.2/anonymous-metadata.patch

+--- /var/www/sharelatex/web/app/src/router.js	2020-09-14 20:21:39.741433000 +0000
++++ /var/www/sharelatex/web/app/src/router.js	2020-09-14 20:13:08.000000000 +0000
+@@ -607,16 +607,17 @@
+     ProjectDownloadsController.downloadMultipleProjects
+   )
+ 
++  console.log(`allowAnonymousReadAndWriteSharing: ${Settings.allowAnonymousReadAndWriteSharing}`)
+   webRouter.get(
+     '/project/:project_id/metadata',
+     AuthorizationMiddleware.ensureUserCanReadProject,
+-    AuthenticationController.requireLogin(),
++    Settings.allowAnonymousReadAndWriteSharing ? (req, res, next) => { next() } : AuthenticationController.requireLogin(),
+     MetaController.getMetadata
+-  )
++  ) 
+   webRouter.post(
+     '/project/:project_id/doc/:doc_id/metadata',
+     AuthorizationMiddleware.ensureUserCanReadProject,
+-    AuthenticationController.requireLogin(),
++    Settings.allowAnonymousReadAndWriteSharing ? (req, res, next) => { next() } : AuthenticationController.requireLogin(),
+     MetaController.broadcastMetadataForDoc
+   )
+   privateApiRouter.post(
+--- /var/www/sharelatex/web/app/src/Features/Contacts/ContactRouter.js	2020-09-14 20:21:52.243779000 +0000
++++ /var/www/sharelatex/web/app/src/Features/Contacts/ContactRouter.js	2020-09-14 20:13:08.000000000 +0000
+@@ -5,6 +5,8 @@
+  * DS102: Remove unnecessary code created because of implicit returns
+  * Full docs: https://github.com/decaffeinate/decaffeinate/blob/master/docs/suggestions.md
+  */
++const Settings = require('settings-sharelatex')
++
+ const AuthenticationController = require('../Authentication/AuthenticationController')
+ const ContactController = require('./ContactController')
+ 
+@@ -12,7 +14,7 @@
+   apply(webRouter, apiRouter) {
+     return webRouter.get(
+       '/user/contacts',
+-      AuthenticationController.requireLogin(),
++      Settings.allowAnonymousReadAndWriteSharing ? (req, res, next) => { next() } : AuthenticationController.requireLogin(),
+       ContactController.getContacts
+     )
+   }

hotfix/2.4.2/left-footer-skip-translation.patch

+
+--- /var/www/sharelatex/web/app/views/layout/footer.pug
++++ /var/www/sharelatex/web/app/app/views/layout/footer.pug
+@@ -32,7 +32,7 @@ footer.site-footer
+ 						if item.url
+ 							a(href=item.url, class=item.class) !{translate(item.text)}
+ 						else
+-							| !{translate(item.text)}
++							| !{item.text}
+ 
+ 			ul.col-md-3.text-right
+ 

hotfix/2.5.1/Dockerfile

+FROM sharelatex/sharelatex:2.5.0
+
+# Patch #826: Fixes log path for contacts service to be picked up by logrotate
+COPY contacts-run.patch /etc/service/contacts-sharelatex
+RUN cd /etc/service/contacts-sharelatex && patch < contacts-run.patch
+
+# Patch #826: delete old logs for the contacts service
+COPY delete-old-logs.patch /etc/my_init.d
+RUN cd /etc/my_init.d && patch < delete-old-logs.patch \
+&&  chmod +x /etc/my_init.d/10_delete_old_logs.sh
+
+# Patch #827: fix logrotate file permissions
+RUN chmod 644 /etc/logrotate.d/sharelatex

hotfix/2.5.1/contacts-run.patch

+--- a/run
++++ b/run
+@@ -7,4 +7,4 @@ if [ "$DEBUG_NODE" == "true" ]; then
+     NODE_PARAMS="--inspect=0.0.0.0:30360"
+ fi
+
+-exec /sbin/setuser www-data /usr/bin/node $NODE_PARAMS /var/www/sharelatex/contacts/app.js >> /var/log/sharelatex/contacts 2>&1
++exec /sbin/setuser www-data /usr/bin/node $NODE_PARAMS /var/www/sharelatex/contacts/app.js >> /var/log/sharelatex/contacts.log 2>&1

hotfix/2.5.1/delete-old-logs.patch

+--- /dev/null
++++ b/10_delete_old_logs.sh
+@@ -0,0 +1,7 @@
++#!/bin/sh
++set -e
++
++# Up to version 2.5.0 the logs of the contacts service were written into a
++#  file that was not picked up by logrotate.
++# The service is stable and we can safely discard any logs.
++rm -vf /var/log/sharelatex/contacts

hotfix/2.5.2/12_update_token_email.js

+const Settings = require('settings-sharelatex')
+const mongojs = require('mongojs')
+const db = mongojs(Settings.mongo.url, ['tokens'])
+const async = require('async')
+
+exports.migrate = (client, done) => {
+  console.log(`>> Updating 'data.email' to lower case in tokens`)
+
+  db.tokens.find({}, { 'data.email': 1 }, (err, tokens) => {
+    if (err) {
+      return done(err)
+    }
+
+    async.eachSeries(
+      tokens,
+      (token, callback) => {
+        db.tokens.update(
+          { _id: token._id },
+          { $set: { 'data.email': token.data.email.toLowerCase() } },
+          callback
+        )
+      },
+      done
+    )
+  })
+}
+
+exports.rollback = (client, done) => done()

hotfix/2.5.2/Dockerfile

+FROM sharelatex/sharelatex:2.5.1
+
+# Patch: fixes registration token creation
+COPY create-token-lowercase-email.patch ${baseDir}
+RUN cd ${baseDir} && patch -p0 < create-token-lowercase-email.patch
+
+# Migration for tokens with invalid email addresses
+ADD 12_update_token_email.js /var/www/sharelatex/migrations/12_update_token_email.js