On Sun, Aug 15, 2021 at 05:35:37PM +0200, Giulio wrote:

> Hello,
> Thank you for you fast reply.
> 
> Il 14/08/2021 23:43, Marek Marczykowski-Górecki ha scritto:

> > 
> > As for the nft syntax, I think iptables-translate tool can help you
> > (part of the iptables-nft package).
> > See https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables
> > 

> > > You can see in [4] how I organized the forwarding mechanism. All the
> > > necessary information, as well as ipv4/ipv6 support should already be in
> > > the 'prepare_forward_rules' function meaning that only the actual
> > > building syntax is left.

> > 

> 
> I have tried to write the external nft rules as well the extarnal ones,
> with the exception of the destination domain.
> 
> Assume the following setup:
> sys-net - 10.137.0.5 (ens6 phy with 192.168.10.20)
> sys-firewall - 10.137.0.6
> personal - 10.137.0.7
> 
> All of them are running fedora-32.
> 
> And assume the following rule added via qvm-firewall:
> # qvm-firewall personal add action=forward forwardtype=external
> scrports=22-22 proto=tcp dstports=2222-2222 srchost=192.168.10.0/24
> .
> First, a table for the forwarding rules is created:
> 
> flush chain {family} qubes-firewall-forward prerouting
> flush chain {family} qubes-firewall-forward postrouting
> table {family} qubes-firewall-forward {
> 	chain postrouting {
> 		type nat hook postrouting priority srcnat; policy accept;
> 		masquerade

I think this is too broad - this will hide the source address of all
incoming connections - something that shouldn't be needed.
masquerade is necessary for outgoing traffic only, but it's there
already in default setup (via iptables...)

> 	}
> 	chain prerouting {
> 		type nat hook prerouting priority dstnat; policy accept;
>         }
> }
> 
> Then, if the qube is marked as 'last', meaning that it is the external
> qube with the physical interface the following rules are added:
> 
> table {family} qubes-firewall-forward {
> 	chain prerouting {
> 		meta iifname "ens6" {family} saddr 192.168.10.0/24 tcp dport {{ 22 }}
> dnat to 10.137.0.6:2222
> 	}
> }
> 
> table {family} qubes-firewall {
> 	chain forward {
> 		meta iifname "eth0" {family} daddr 10.137.0.6 tcp dport 2222 ct state
> new counter accept

iifname "eth0" ? Should be rather ens6.

> 	}
> }
> 
> And that is all for sys-net.
> 
> In sys-firewall, since it is an 'internal' qube, the following rules are
> added instead:
> 
> table {family} qubes-firewall-forward {
> 	chain prerouting {
> 		meta iifname "eth0" {family} saddr 120.137.0.5 tcp dport {{ 2222 }}
> dnat to 10.137.0.7:2222

And here, if there wouldn't be masquerade for everything, you could keep
the original source addr (192.168.10.0/24)

> 	}
> }
> 
> table {family} qubes-firewall {
> 	chain forward {
> 		meta iifname "eth0" {family} daddr 10.137.0.7 tcp dport 2222 ct state
> new counter accept
> 	}
> }
> 
> Lastly, the appropriate rules allowing incoming traffic on the selected
> port from the previous hop should be added directly yo the 'personal'
> domain. However I see that there the nft ruleset is empty, while
> iptables seems indeed to be in use. I guess that those rules are the
> ones specified in qubes-core-agent-linux/network/iptables, however I am
> wondering how we should proceed on this one?

Ok, this indeed is an issue with mixed iptables / nft usage. For the
purpose of this project, since there isn't much time left, you can
simply stop 'iptables' service in the 'personal' VM - and document
this as a manual step needed. This will become unnecessary when iptables
rules will be migrated to nft. 

But there is also another issue: the qubes-firewall daemon is currently
not started if a VM doesn't provide network. So, it isn't started in
'personal' VM here. 

> Also are you able to spot errors or something missing in the
> aforedescribed rule flow? When testing I can see the incoming connection
> on port 22 of the physical interface of sys-net, but then I am losing
> track of the connection after that...

See above - the interface name. You may also like to see this:
https://wiki.nftables.org/wiki-nftables/index.php/Ruleset_debug/tracing

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab