Hi, I'm replying to both emails at once:

On Sun, Jun 20, 2021 at 10:50:04PM +0200, Giulio wrote:

> Hello,
> sorry for the late reply.
> 
> I read a lot of code and I have to admit that I did not grasp the
> complexity of the Admin API and networking stack before looking so much
> into it. I think I've got the overall picture, but it will take a little
> more to fully be confident moving there.

(...)

> _core-agent-linux/qubesagent/firewall.py_
> Is the actual file responsible for running nftables and thus
> adding/deleting/reloading firewall ruless in the target firewall vm. It
> also resolves DNS names for domain rules. It is run by Admin API (qubesd).

This file is run as part of the qubes-firewall service in relevant VMs
(especially sys-firewall).

> _manager/qubesmanager/firewall.py_
> Contains the code for the "Firewall" tab of the "Qube Manager" window.
> 
> _manager/ui/qubemanager.ui_
> String and properties for the "Qube Manager" UI.
> 
> 
> Questions:
> 
> 1) Should we both support internal port forwarding and external port
> forwarding? Such as exposing a port for another domain or exposing a
> port through the public network interface? I would say yes.

Yes, I think so. Technically, those two cases should be quite similar.
See also the case of sys-vpn much lower in the email.

> 2) Should it be possible to add rules with an 'any' clause (both tcp and
> udp). I would say no because since port forwarding brings a higher
> attack surface all rules should be as precise as possible.

Indeed. Furthermore, most services are either TCP or UDP, very few are
both (and for those, it's ok to require two rules).

> 3) Since the expire= feature seems to be already implemented (and
> limited for the expiring full outgoing access) would it be useful to be
> implemented in gui and cli for every rule? I would say yes since the
> admin and agent code seems to be already there. The same goes for the
> "comment=" field.

Per-rule expire may be tricky to handle at the GUI level, I have no idea
how to make the UI for this not very confusing...
But the comment field is definitely useful to use.

> 4) How would you implement the management of forwarding rules in the
> network providing domain (sys-net)? Shall the user add a rule both in
> the target domain (ie the one with webserver and another one in sys-net)
> or should it be fully automatic from the first?

From the user point of view, I think it should be automated as much as
possible. Like, let the user choose which port in which VM redirect to
where. There may be cases when such redirection won't be possible - if
there is no network path between the two points.

> 5) Users should be able to set forward rules using domain names and not
> static ip addresses. In this case, the actual ip addresses of the dst
> domains should be collected in a similr way as currently DNS are
> resolved in `/core-agent-linux/qubesagent/firewall.py`, would this be good?

But here we are mostly talking about IP addresses of different VMs,
right? Those can (and should) be resolved at core-admin side, so the VM
applying the rules will have all the IP given. In fact VM may not be
able to resolve IP of another VM at all.

> Proposed XML Syntax:
> <rule>
> 	<properties>
> 		<property name="action">forward</property>
> 		<property name="proto">udp</property>
> 		<property name="dstports">443-8080-5555</property>
> 	</properties>
> <rule>

I don't see an important information here: forward to _where_.

> Proposed Admin API Syntax:
> action=forward proto=udp dstports=443-8080-5555 [expire=<unix
> timestamp>] [comment=random text]

Similar here, there needs to be a forward target (IP, and possibly a
port)

> 
> I also plan to document, at least partially, the journey into this.
> As a last question, I'm curious what is your setup in order to test
> modifications in the aforementioned repos while developing.

In practice, I have a separate (physical) computer to run various tests.
It can be a system installed on an external disk. But there are also
other options:
1. Test on the system you are developing on - this may be the easiest
option, but is also a risky one (it is easy to break things - may be
undesirable if you need that system for other stuff too).
2. Test in a virtual machine - it's possible to run Qubes OS inside KVM
with nested virtualization enabled (included emulated IOMMU - sadly
works only Intel only). This requires non-Qubes host. While it is
possible to run Qubes inside Qubes, the setup is quite challenging and
fragile, so I don't recommend this path.
3. As a last resort, I have a bunch test systems and I can give you
access to one of them via SSH. But developing network-related changes,
accessing the system through network, may not be the wisest thing to
do...

You can also find some helpful info here:
https://www.qubes-os.org/doc/#debugging

On Tue, Jun 22, 2021 at 01:49:15AM +0200, Giulio wrote:

> Hi,
> sorry for yesterday long and a bit confusing message. I started writing
> down my documentation and progress here https://git.lsd.cat/Qubes/gsoc
> so it should me more readable and easier to follow.

No worries. Some comments for this page:

> qvm-firewall <vmname> add action=accept dsthost=1.1.1.1 proto=tcp dstports=80-80 command="cloudflare http test rule" expire=+5000

It's "comment", not "command". And also, if you ever need to write Admin
API level line manually, the comment field must be last.

(but on qvm-firewall cmdline it is fine as is)

> Since in the case of port forwarding the target ip address would always be the <vmname> IP address,

This is very true. But there needs to be an information where to forward
the traffic to (as noted earlier). Plus, possibly a second set of ports
(if you want to redirect to a different port).

> I think my main concern now is the question 4 from the aforementioned
> email. Shall the rules be automatically implemented in all 3 involved
> vms? (<netvm,firewallvm,appvm>). I think yes, because otherwise it would
> be counterintuitive to be a partially manual and partially automatic
> operation. But since it actually 'automatically' exposes more attack
> surface, by loosening up all 3 vms network rules, I guess that maybe
> more reasoning on it would be helpful.

Yes, but you need to pass the traffic somehow. The direct connection can
be achieved with qvm-connnect-tcp (connecting to the target directly
using qrexec, bypassing intermediate VMs), but it has its limits (low
performance, TCP only). To keep it as actual IP traffic, you need to
change firewall rules at all intermediate VMs too.

Lets have a specific example: in default setup, redirect TCP port 80
from the outside, to 'work' VM port 8080. 

The setup looks like this:
  
  sys-net -> sys-firewall -> work

For this, you will need those rules:

1. In sys-net: forward TCP port 80 to sys-firewall
2. In sys-firewall: forward TCP port 80 to work, port 8080
3. In work: allow TCP port 8080

Now is the important design question: how to store those rules? If you
store them at all three places separately, it
will be easier to apply them at runtime, but it will be harder to
correlate them in UI. Plus, if any of them get modified/removed, it may
be non-trivial to troubleshoot the issue.
The other approach is to store the forward rules only in one place (the
target, 'work' in this example? or the source, 'sys-net' here?). This
way, it's harder to mess thing up. But when applying the rules (building
rule sets for qubes-firewall service in all the involved VMs), you need
to check several places.
Plus, the UI should clearly show such redirected ports at all involved
places, because it does affect system security - it must be easy to spot
if any redirects are enabled.


To make things more complex (sorry...), there may be a VPN or other
proxy service (Tor?) involved. For example:

sys-net -> sys-firewall -> sys-vpn -> work

In such a case, the "external" VM for 'work' is not really sys-net, but
rather sys-vpn. And actually you need to be careful to not accidentally
bypass VPN either by allowing 'work' to communicate outside of the VPN,
or (maybe even worse) systems on the LAN (via sys-net) reach inside VPN.

This case is not easy to solve, because currently core-admin has no idea
whether sys-vpn (or other such VM) do any of such tunnelling. Maybe we
need to (finally) introduce some flag to mark such VMs?


And another question: what should happen if you change netvm of 'work'.
For example switch to something like:

    sys-net -> sys-firewall -> (other VMs, but not 'work')

    sys-wifi -> work

Should the redirection stay active via sys-wifi? I think it should not,
at least not automatically (maybe have an option for that?).


And finally, don't forget IPv6 exists. Which means you can have the same
port on IPv4 and IPv6. And theoretically they could be redirected to
different places (but I'm not sure if that's a good idea...).

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab