As the person who actually does most of the backend stuff including trying to manage the lag, I feel I should weigh in here.
-The server package downgraded slightly a while ago, but this is unlikely to be the culprit. Upgrading it again is a possibility, but I don't think it'd solve much.
Firstly, the downgrade was over a year ago, and as far as I can remember it was a minor change, I think -0.5GB RAM. Everything else about the server remained the same. I highly doubt that is the cause, considering the server didn't lag until about 8 months later when we upgraded to 1.14. Also, from all my tests, it seems that being out of memory very rarely happens, if ever, so I'm not sure what good adding that back in would do. Changing the server package was done more to save money rather than to be a downgrade. When me and Ako were discussing it, it was decided that the very small drop in specs was an acceptable loss, considering that extra bit was rarely in use, yet that (combined with some other tweaks) enabled us to reduce our monthly expenditure, which helps keep the server running longer. It's expensive to run this, simply saying "let's upgrade the server" means an additional cost every month, which may not come through in donations.
-Backend update work is being done and more will be done soon to improve performance.
There is a lot of backend work that never gets seen. Every couple of days I'm looking at logs of the server and seeing how it's performing, trying to decipher what is causing problems and trying to fix it. Just last week I tweaked some memory settings which improved the stability of the TPS quite a bit, and I tweaked them some more yesterday which seems to have had another beneficial effect. I've been doing a lot of profiling of the server, where I monitor every single process that happens over a few hours and get a massive long tree of everything and how much lag it's contributing, and use these to try and make those processes more efficient. I'm constantly changing settings, tweaking things here and there, updating stuff, patching stuff, etc. Lag doesn't just come from a single source, saying "farms cause all the lag" is wrong. Lag comes from millions of tiny things hidden around everywhere, which all add up over time to be a noticeable effect. Every time I profile it I get slightly different readings, which is expected, because the conditions are never the same. Sometimes the server has lots of people on, sometimes it's quiet, sometimes people are at farms, sometimes people are building. But looking at a range of these does give an overview of things that tend to take more time than the rest, and these are what I try and focus my efforts on.
Recently there has been an increase in server lag, likely caused by the use of large farms while other people are on the server.
Farms have always been around on VC, you'd have to try really hard to find any prolonged period of time in the past few years where farms weren't used. They haven't suddenly started causing lag recently, and getting rid of them isn't the end-all solution to lag. There's other things at play which cause it. If you look at
this chart of TPS over the past month, you can see it fluctuates every minute of every day, and the same pattern appears roughly every month. Don't try and pin this lag on farms which have been here for years.
-Mojang are a weird group of folks and Minecraft is a bizarrely built game.
Mojang's only quality control is us, and 1.14 was a rushed version with very few bug fixes in the snapshots. It's an awful update in terms of performance, and literally any other billion dollar company would've shot that down the second it arrived at their QA/QC teams. From what I can tell, 1.16 will be a lot more stable, and I'm very eager to update to it as soon as is possible following the update, even if that means re-coding problematic plugins myself.
There is nothing in the rules saying you cant use them w/ other people on. However, many farm designs cause lag on the server, and it is inconsiderate to everyone else on server if you generate lag by afking a farm, while other people are trying to play.
Being inconsiderate is not against the rules. No, it might not be the best idea, but they're not doing anything wrong. They're playing and enjoying the game just like you. Why should you say to them they can't do what they want in the game in order to let someone else do so?
On that note, I believe the biggest problem with big farms is that people tend to build them where they spend a lot of time. While this makes sense from the perspective of using a large farm to make money, the problem is that with the game becoming less optimized in 1.14, these farms generate lag that can make it difficult to play on the server.
I am not the first to admit that I personally have a very large farm. however, besides taking measures to mitigate the lag it generates, I also built it far far away from anything else, such that if I want to use it, I deliberately have to go afk it. For the region around it where I can activate the farm, I dont spend any time. This helps mitigate the lag the farm causes.
I think if people want to build large farms, its fine, even if the farm tends to cause a lot of lag. However, build it far away from populous areas, and avoid being nearby when there are other people on server trying to enjoy minecraft. If people who want to build large farms take these steps, I think we can have a much less laggy server, without having to spend money on a better server package.
By moving farms away from where they currently spend a lot of time, it will eventually just move where people spend a lot of time. It's not a great solution to the problem. Plus, as I said, getting rid of farms isn't a magical solution to the lag.
TL;DR: Getting rid of farms won't magically solve problems with lag. It's caused by lots of tiny things, which I spend a significant amount of my time trying to find ways of improving performance. Speculating on things and attacking each other for stuff without strong evidence isn't good, so please stop doing it. There's lots of things going on behind the scenes to reduce lag, but it's not going to be as simple as "ban farms".
Oh also having stayed up until 0300 watching you all squabble over lag I ended up waking up at around 1230 today, so thanks for making today unproductive