nfs mounting not working

[William Unruh]

(all names and addresses have been altered, but the form is the same.)

I am trying to mount a directory from Serv to Client

On Cient I have in /etc/fstab
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
and /local1 exists as an empty directory.

On Serv, I have
/local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
in /etc/exports
where the address of Client is 142.103.222.11
and I ran exportfs -s
The directory /local exists on Serv.
ls -dl /local
drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/

But when I type
mount /local1
on Client as root I get
mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory

I cannot find anything in any of the /var/log files which gives me a clue
as to what could be going on here.

If on Client I change the entry to
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
I then get an entry in journalctl of
Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
But now the error message on Clinet is
mount.nfs: access denied by server while mounting Serv:/local/

Ie, nfs seems to be all F up, but I cannot figure out why.

[William Unruh]

Note that this all happened after I rebooted Serv. nfs has been working
fine until recently. and all of the machines had their nfs mounts
working. The problem started when I rebooted Client and nothing was
mounted.
Note that all of the directories mounted from Serv onto Client are
affected. The case below is just an example.

All are on updated Mga8 systems.

Post by William Unruh
(all names and addresses have been altered, but the form is the same.)
I am trying to mount a directory from Serv to Client
On Cient I have in /etc/fstab
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
and /local1 exists as an empty directory.
On Serv, I have
/local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
in /etc/exports
where the address of Client is 142.103.222.11
and I ran exportfs -s
The directory /local exists on Serv.
ls -dl /local
drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/
But when I type
mount /local1
on Client as root I get
mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory
I cannot find anything in any of the /var/log files which gives me a clue
as to what could be going on here.
If on Client I change the entry to
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
I then get an entry in journalctl of
Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
But now the error message on Clinet is
mount.nfs: access denied by server while mounting Serv:/local/
Ie, nfs seems to be all F up, but I cannot figure out why.

[William Unruh]

And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Post by William Unruh
Note that this all happened after I rebooted Serv. nfs has been working
fine until recently. and all of the machines had their nfs mounts
working. The problem started when I rebooted Client and nothing was
mounted.
Note that all of the directories mounted from Serv onto Client are
affected. The case below is just an example.
All are on updated Mga8 systems.

Post by William Unruh
(all names and addresses have been altered, but the form is the same.)
I am trying to mount a directory from Serv to Client
On Cient I have in /etc/fstab
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192 0 0
and /local1 exists as an empty directory.
On Serv, I have
/local 142.103.222.0/26(fsid=21,no_subtree_check,rw,no_root_squash)
in /etc/exports
where the address of Client is 142.103.222.11
and I ran exportfs -s
The directory /local exists on Serv.
ls -dl /local
drwxrwxr-x 22 root root 4096 Sep 18 2022 /local/
But when I type
mount /local1
on Client as root I get
mount.nfs4: mounting Serv:/local/ failed, reason given by server: No such file or directory
I cannot find anything in any of the /var/log files which gives me a clue
as to what could be going on here.
If on Client I change the entry to
Serv:/local/ /local1 nfs4 rw,rsize=8192,wsize=8192,vers=3 0 0
I then get an entry in journalctl of
Serv rpc.mountd[775]: authenticated mount request from 142.103.222.11:958 for /local/unruhhome (/local/unruhhome)
But now the error message on Clinet is
mount.nfs: access denied by server while mounting Serv:/local/
Ie, nfs seems to be all F up, but I cannot figure out why.

[David W. Hodgins]

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?

Regards, Dave Hodgins

[William Unruh]

Post by David W. Hodgins

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?
Regards, Dave Hodgins

On Serv:

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0

On Client:
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0

I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

[David W. Hodgins]

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?
Regards, Dave Hodgins

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0
I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

The dropped packet count is very high for that number of packets.

On my system after 2 days since reboot ...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
12995214891 24029134 0 0 0 7143
TX: bytes packets errors dropped carrier collsns
12474040445 27377647 0 0 0 0

All three of my computers, two of which use ethernet and the other wireless have
zero dropped packets.

The most likely causes of packet cause is one or more bad ethernet cables, bad
sockets, or just not quite being fully plugged into the socket.

Regards, Dave Hodgins

[William Unruh]

Post by David W. Hodgins

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?
Regards, Dave Hodgins

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0
I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

The dropped packet count is very high for that number of packets.
On my system after 2 days since reboot ...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
12995214891 24029134 0 0 0 7143
TX: bytes packets errors dropped carrier collsns
12474040445 27377647 0 0 0 0
All three of my computers, two of which use ethernet and the other wireless have
zero dropped packets.
The most likely causes of packet cause is one or more bad ethernet cables, bad
sockets, or just not quite being fully plugged into the socket.

Thanks for the hint. It wou.d be strange if such a cause could reliably
result in "directory name does not exist" as the message from the server
to the client, while ssh connections between the two seems to be fine.

Post by David W. Hodgins
Regards, Dave Hodgins

[David W. Hodgins]

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?
Regards, Dave Hodgins

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0
I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

The dropped packet count is very high for that number of packets.
On my system after 2 days since reboot ...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
12995214891 24029134 0 0 0 7143
TX: bytes packets errors dropped carrier collsns
12474040445 27377647 0 0 0 0
All three of my computers, two of which use ethernet and the other wireless have
zero dropped packets.
The most likely causes of packet cause is one or more bad ethernet cables, bad
sockets, or just not quite being fully plugged into the socket.

Thanks for the hint. It wou.d be strange if such a cause could reliably
result in "directory name does not exist" as the message from the server
to the client, while ssh connections between the two seems to be fine.

I'm not sure if that is the cause or not. I haven't looked into how nfs/nfsd
and ssh/sshd handle some failed packets. It wouldn't surprise me if they work
differently, causing the problem only for nfs. The server may not be generating
that message. It may be coming from the client whenever the mount fails.

I'd try fixing that issue and then see if the problem still occurs.

Regards, Dave Hodgins

[William Unruh]

Thanks will try that if it occurs again. The reason I suspect the server
is that it occured to at least two clients. On the one client I guess it
had been rebooted and had mounted nothing from the server. On the second
client, it had not been rebooted, and nfs worked fine, until I unmounted
one of the partition, and then on remount, it had the same problem.

I finally and accidentally did something to the server, and everything
started working again ( the clients all remouted without problems). of
course I did not notice what I did.

Post by David W. Hodgins

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh

Post by David W. Hodgins

Post by William Unruh
And of course just after I sent that out, the systems suddenly started
working again. Not at all clear what happened.

Does "ip -s link" show any errors?
Regards, Dave Hodgins

2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether f8:32:e4:70:14:5b brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
69357175201 558088800 0 37942 0 15654970
TX: bytes packets errors dropped carrier collsns
1395857318948 985313209 0 0 0 0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 20:cf:30:90:5c:cb brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
715899055 1004543 0 1095 88 83900
TX: bytes packets errors dropped carrier collsns
379358269 614341 0 0 0 0
I have no idea what time span those results cover, and the mount
problems occured abot 5 days ago.

The dropped packet count is very high for that number of packets.
On my system after 2 days since reboot ...
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether 10:fe:ed:03:8d:b3 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
12995214891 24029134 0 0 0 7143
TX: bytes packets errors dropped carrier collsns
12474040445 27377647 0 0 0 0
All three of my computers, two of which use ethernet and the other wireless have
zero dropped packets.
The most likely causes of packet cause is one or more bad ethernet cables, bad
sockets, or just not quite being fully plugged into the socket.

Thanks for the hint. It wou.d be strange if such a cause could reliably
result in "directory name does not exist" as the message from the server
to the client, while ssh connections between the two seems to be fine.

I'm not sure if that is the cause or not. I haven't looked into how nfs/nfsd
and ssh/sshd handle some failed packets. It wouldn't surprise me if they work
differently, causing the problem only for nfs. The server may not be generating
that message. It may be coming from the client whenever the mount fails.
I'd try fixing that issue and then see if the problem still occurs.
Regards, Dave Hodgins

[David W. Hodgins]

Post by William Unruh
Thanks will try that if it occurs again. The reason I suspect the server
is that it occured to at least two clients. On the one client I guess it
had been rebooted and had mounted nothing from the server. On the second
client, it had not been rebooted, and nfs worked fine, until I unmounted
one of the partition, and then on remount, it had the same problem.
I finally and accidentally did something to the server, and everything
started working again ( the clients all remouted without problems). of
course I did not notice what I did.

I still recommend figuring out why so many packets are being dropped. Even
if the nfs mounts are working again now, the networking problems are likely
to cause additional problems later.

In addition to checking the cables and sockets, is the time being synced
properly on all of the systems?

Regards, Dave Hodgins