How To Configure YaCy
YaCy with abundant resources is stable, but not everyone has unlimited resources.
This is the beginners guide to configuring YaCy as I have done it. If you are an experienced user or a JAVA developer, this is probably not what you are looking for. This guide is meant for those who have just installed YaCy and want to get a handle on the settings required to make YaCy happy – or keep it from crashing.
YaCy version: 1.924/10042
I will try to explain in simple English how to set resources so that YaCy runs smoothly.
My first few goes with YaCy resulted in crashes due to my poor understanding what these settings should be, relative to the resources of the machine. Which bring me to the obvious.
Running YaCy from a laptop or small personal computer is fine if you are just giving YaCy a spin, or if you are not going to crawl much, like an Intranet. But if you are going to build any kind of real search index specific to your interests, I recommend getting a VPS straight-away. You’ll be happy you did.
There are 2 menus where all settings we will concern ourselves with at this point are controlled. Both pages are under RAM/Disk Usage & Updates
/Performance_p.html
RAM/Disk Usage & Updates > Performance > Memory Settings > Memory Reserved for JVM
The Memory Reserved for JVM is the main working memory. When you set this reserve RAM, keep in mind that the setting at the bottom (RAM – minimum RAM) is the bare minimum and you don’t want these two to conflict. So if you set Memory Reserved for JVM to use most of your available RAM, you may cause YaCy to stop communicating (DHT) when it consumes so much free RAM that it hits your minimum RAM limit. This conflict in memory settings may be the single most common problem you will encounter – so pay attention to this. This also explains why personal computers without adequate RAM are problematic (inappropriate for YaCy).
Find how much RAM YaCy is using in the GUI (browser) > System Status > Memory Usage > RAM Used
This makes it pretty easy to see how much RAM is available and how much is currently Used. Adjust accordingly.
I also like to use HTOP (linux) at the same time to give a clear picture of system resources.
I found that with 48Gb total RAM on my server, I reserved 24Gb for JVM (YaCy only uses between 5Gb – 14Gb). This setting seems to work well. I have played around reserving more and less (down to 10Gb) and YaCy never complained.
RAM/Disk Usage & Updates > Performance > Resource Observer > Free Space Disk
Find how much disk space you have free in the GUI (browser) > System Status > Disk Free
This setting is the MINIMUM disk space remaining free before YaCy shuts down. OK, it doesn’t shutdown entirely, but that is effectively what happens when the drive has less than this much space remaining free. If YaCy hits this limit, it will appear that YaCy is not doing anything, which is essentially true. Get a bigger disk!
RAM/Disk Usage & Updates > Performance > Resource Observer > Used Space Disk
You can see how much Disk space YaCy is using in the GUI (browser) > System Status > Disk Used
This setting is the MAXIMUM disk space you want to allow YaCy to use. If the actual Disk space used hits this setting, it doesn’t shutdown entirely, but that is effectively what happens when the drive has less than this much space remaining free. If YaCy hits this limit, it will appear that YaCy is not doing anything, which is essentially true. You won’t be able to index anything new and you won’t be exchanging RWI (DHT) with peers. Lower the setting or get a bigger disk.
RAM/Disk Usage & Updates > Performance > Resource Observer > RAM
The MINMUM RAM setting default is fine. Orbiter says YaCy does not need more, but that might depend on how much RAM other programs on your machine require.
Lets repeat what I said above; if you set Memory Reserved for JVM to use most of your available RAM, you may cause YaCy to stop communicating (DHT) when it consumes so much free RAM that it hits your minimum RAM limit. This conflict in memory settings may be the single most common problem you will encounter – so pay attention to this. This also explains why personal computers without adequate RAM are problematic (inappropriate for YaCy).
/ConfigHTCache_p.html
RAM/Disk Usage & Updates > Web Cache
This is again a Disk space setting. So if you got it, use it. The cache available will speed up crawling/indexing.
That it. ! YaCy runs smoothly when these settings match the resources available on you system ! But remember, watch the logs.