Server troubles

[Stu]

You all may have noticed a small check that has been happening before being allowed on to the forum

This is because I have having some issues with spam bots/scrappers hitting the forum and causing high load which in turn takes the forum offline!

I have currently turned it off as I am trying something to see if I can stop it happening but if it starts again I will need to turn the check back on until I get to the bottom of it!

[burridge01]

I noticed that Stu,not bothered by it.If it helps the forum then it is what it is

[Stu]

The plan is to find a way to block whatever is accessing the forum!

Seems like a rogue bot to me

[EasyRider]

Lots of websites are doing it now. I think it a good thing and worth having to stop the bots and spam.
But i notice "Bing" & "Google" Bots are still showing as users online.

[Stu]

Yes there is a bigger issue to this and its since all these AI systems started to come out!

I have enabled the bot check again but adjusted the settings so its less intrusive to everyday users so its there working in the background just less obvious

One of my forums had over 20k visitors in one go and this about 9k which brought the server down again!

[Doctor T]

Why, with every post, does the system ask if I'm a human. I may not be normal, but I know i belong to this race. I've signed out and in, but the result is the same.

[Stu]

Shouldn't be doing it on every post!

[Doctor T]

Every post Stu, even with a thumbs up. I've been using a Chromebook for over 2 years, and this has just started

[Stu]

You got any add blockers or something like that running?

[Stu]

Just to put some context as to why you are asked if you are human or not is since the big take up in AI there is a lot of Bots that are visiting websites and crawling them for information which is not necessarily a bad thing but its the amount that are visiting that is the real issue and there is what are called bad bots too which serve no purpose at all.

These bots come in their hundreds of thousands of visits which eats up the bandwidth and server resources thus taking down the server and making it unreachable/unresponsive which gives real users a bad experience.

Here is a screenshot on the last 30 days of unique visitors to the forum which you can clearly see where I turned on the human check feature.

This is daily visits and that high peak is almost 1 million unique visitors per day. Although the line flattens off and looks really low for the rest of the 30 days that is still around 10-12k unique visitors per day which are all real humans or good bots.

Forum Stats.png (86.56 KiB) Viewed 6812 times

This is all about trying to have a good end user experience without it being too intrusive.

I have the setting set to only check on a weekly basis but due to rolling IP addresses and network connections been reset this can be more frequent.

I use a third party DNS called Cloudflare which helps to protect against things like this and they are working hard to make improvements in the background so website owners can turn the are you human check off but this takes time.

I will periodically turn it off to see how the website performs but if it starts to get overloaded again then I have no choice but to turn it back on again.

I know it can be a little annoying but please bear with it and hopefully a permanent solution is found.

[Doctor T]

No problem Stu. Yes, it is Cloudflare that is asking me if I am human.

[owl]

You're doing a great job. A momentary irritation logging in is nothing.