How To Debug YaCy

by · Published · Updated

This is the beginners guide to debugging YaCy as I have experienced it. This guide is meant for those who have just installed YaCy and need a little help. YaCy NOOBS may not even be able to find the right question to ask in the forum. So I will try to fill that gap here;

  1. YaCy seems to be running, but nothing is indexing.
  2. YaCy will not start. (or it crashes immediately after start)

Logs

The GUI is nice when things are running normally, but oddly the GUI has no Notifications when things are not working. And of course, if YaCy wont start, how do you access the GUI in the browser?

So the first thing to do – look at the logs. YaCy’s logging is very good. You will probably find what is wrong with a quick look at the last log file (yacy00.log).

In your terminal type (assuming you installed YaCy on linux and in your home directory, replace “user” with your username)

tail -f /home/user/yacy/DATA/LOG/yacy00.log

This is your first go-to anytime – even when YaCy is running normally, this will tell you what is happening or what YaCy got hung up on.

If there are lots of lines scrolling rapidly past, chances are everything is fine. YaCy log will tell you about the peers its communicating with and what the crawler and indexer are doing – you just have to read really really fast!

If on the other hand, if lines are stationary, chances are there is a problem, and you should be able to spot the line that tells you what is wrong by scrolling up. If scrolling up doesn’t reveal anything obvious. Hit ctrl-c and type;

grep -r 'failed' /home/user/yacy/DATA/LOG/

A few things you should grep for:

  • failed
  • write.lock

YaCy logs are verbose and rotate quickly. YaCy maintains 20 log files in that directory. That is why the grep command above looks at all the log files. But you may want to start by only looking at yacy00.log, (most recent) then yacy01.log and so on.

ls /home/user/yacy/DATA/LOG/ -latr

YaCy is running (you can access the GUI in the browser)

If you do find the problem, chances are its an index problem. There are a few things that seems to go wrong over time and seemingly the most reliable solution (in the docs) is to simply restart YaCy.

But wait! You want to Restart or Shutdown from within the browser GUI. The reason for the distinction is that using the command line stopYACY.sh command is not always effective. That is my experience (and one day I’ll take it up in the forum). However the GUI Shutdown (button) seems to work every time.

YaCy won’t start (you cannot access the GUI in the browser)

Watch the tail log on yacy00.log while running startYACY.sh. If YaCy doesn’t start, the error should be right in front of you.

Take a look at HTOP (linux) while running startYACY.sh. Is RAM the problem?

I have had a corrupt index occur and while I tried finding solutions in the forum, my solution, as unwelcome and extreme as it was, was to delete the entire index. And then upon restarting, everything was 100%. (I just had to re-crawl a lot of websites)

I believe my mistake, which caused the index to break, was attempting to shutdown using stopYACY.sh. When it failed to actually shutdown YaCy after a couple tries, frustrated I rebooted linux. Bad behavior begets bad things.

Best Practice: Always Shutdown or Restart YaCy using the GUI (browser)

If adjusting the resource settings is required, but you can’t get to the GUI, you can edit setting using your favorite text editor (linux)

sudo vi /home/user/yacy/DATA/SETTINGS/yacy.conf

There are several resource settings (re. resource.disk.used.max.steadystate=500000)

Adjust and try restarting.

Those settings ( configuration file yacy.conf ) can be seen/set in the GUI > System Administration > Advanced Properties

Tags:

You may also like...