Hello,
thank you for the detailed response.

Il 22/06/2021 04:43, Marek Marczykowski-Górecki ha scritto:

Hi, I'm replying to both emails at once:

On Sun, Jun 20, 2021 at 10:50:04PM +0200, Giulio wrote:

Questions:

1) Should we both support internal port forwarding and external port
forwarding? Such as exposing a port for another domain or exposing a
port through the public network interface? I would say yes.

Yes, I think so. Technically, those two cases should be quite similar.
See also the case of sys-vpn much lower in the email.

I think that I'm actually failing to picture all the possible internal scenarios.

1) In the case of external port forwarding <sys-net> should forward to <sys-firewall> and <sys-firewall> then to the <appvm>.
In this case the port gets forwarded on the external interface ie: a LAN or a public ip address depending on the network environment.
2) In the case of internal port forwarding, the port is forwarded only from <sys-firewall> to <appvm>. In that case, another <appvm2> can visit the <appvm> service using <sys-firewall> ip address and the chosen port.
In this case, the ports get exposed on <sys-firewall> and thus depending on how the rules are implemented, may be available to all the AppVMs that share the same <sys-firewall>.

In both cases may be important to allow to specify access rules for the forwarded port, such as the lan/public ip addresses ranges allowed for case 1 and the appvm name for case 2.

3) Since the expire= feature seems to be already implemented (and
limited for the expiring full outgoing access) would it be useful to be
implemented in gui and cli for every rule? I would say yes since the
admin and agent code seems to be already there. The same goes for the
"comment=" field.

Per-rule expire may be tricky to handle at the GUI level, I have no idea
how to make the UI for this not very confusing...
But the comment field is definitely useful to use.

How do you see the same checkbox that actually allows full internet access with the 5 minutes expiration time, displayed also on the window for adding a rule?
However I think there is time to think more through this as the UI will be the last component.

4) How would you implement the management of forwarding rules in the
network providing domain (sys-net)? Shall the user add a rule both in
the target domain (ie the one with webserver and another one in sys-net)
or should it be fully automatic from the first?

From the user point of view, I think it should be automated as much as
possible. Like, let the user choose which port in which VM redirect to
where. There may be cases when such redirection won't be possible - if
there is no network path between the two points.

I agree with you. We might just check when the user adds an internal forwarding rule if both the source and the destination shares the same <firewallvm>, don't we?

5) Users should be able to set forward rules using domain names and not
static ip addresses. In this case, the actual ip addresses of the dst
domains should be collected in a similr way as currently DNS are
resolved in `/core-agent-linux/qubesagent/firewall.py`, would this be good?

But here we are mostly talking about IP addresses of different VMs,
right? Those can (and should) be resolved at core-admin side, so the VM
applying the rules will have all the IP given. In fact VM may not be
able to resolve IP of another VM at all.

Thanks for the insight, it totally makes sense.

Proposed XML Syntax:
<rule>
    <properties>
        <property name="action">forward</property>
        <property name="proto">udp</property>
        <property name="dstports">443-8080-5555</property>
    </properties>
<rule>

I don't see an important information here: forward to _where_.

Proposed Admin API Syntax:
action=forward proto=udp dstports=443-8080-5555 [expire=<unix
timestamp>] [comment=random text]

Similar here, there needs to be a forward target (IP, and possibly a
port)

On Tue, Jun 22, 2021 at 01:49:15AM +0200, Giulio wrote:

Since in the case of port forwarding the target ip address would always be the <vmname> IP address,

This is very true. But there needs to be an information where to forward
the traffic to (as noted earlier). Plus, possibly a second set of ports
(if you want to redirect to a different port).

I am still failing to understand something here, could you give me a an example on when the dsthosts would different rather than the <appvm> or <firewallvm> ip?

I think my main concern now is the question 4 from the aforementioned
email. Shall the rules be automatically implemented in all 3 involved
vms? (<netvm,firewallvm,appvm>). I think yes, because otherwise it would
be counterintuitive to be a partially manual and partially automatic
operation. But since it actually 'automatically' exposes more attack
surface, by loosening up all 3 vms network rules, I guess that maybe
more reasoning on it would be helpful.

Yes, but you need to pass the traffic somehow. The direct connection can
be achieved with qvm-connnect-tcp (connecting to the target directly
using qrexec, bypassing intermediate VMs), but it has its limits (low
performance, TCP only). To keep it as actual IP traffic, you need to
change firewall rules at all intermediate VMs too.

Lets have a specific example: in default setup, redirect TCP port 80
from the outside, to 'work' VM port 8080.

The setup looks like this:
      sys-net -> sys-firewall -> work

For this, you will need those rules:

1. In sys-net: forward TCP port 80 to sys-firewall
2. In sys-firewall: forward TCP port 80 to work, port 8080
3. In work: allow TCP port 8080

Now is the important design question: how to store those rules? If you
store them at all three places separately, it
will be easier to apply them at runtime, but it will be harder to
correlate them in UI. Plus, if any of them get modified/removed, it may
be non-trivial to troubleshoot the issue.
The other approach is to store the forward rules only in one place (the
target, 'work' in this example? or the source, 'sys-net' here?). This
way, it's harder to mess thing up. But when applying the rules (building
rule sets for qubes-firewall service in all the involved VMs), you need
to check several places.
Plus, the UI should clearly show such redirected ports at all involved
places, because it does affect system security - it must be easy to spot
if any redirects are enabled.


To make things more complex (sorry...), there may be a VPN or other
proxy service (Tor?) involved. For example:

sys-net -> sys-firewall -> sys-vpn -> work

In such a case, the "external" VM for 'work' is not really sys-net, but
rather sys-vpn. And actually you need to be careful to not accidentally
bypass VPN either by allowing 'work' to communicate outside of the VPN,
or (maybe even worse) systems on the LAN (via sys-net) reach inside VPN.

This case is not easy to solve, because currently core-admin has no idea
whether sys-vpn (or other such VM) do any of such tunnelling. Maybe we
need to (finally) introduce some flag to mark such VMs?


And another question: what should happen if you change netvm of 'work'.
For example switch to something like:

     sys-net -> sys-firewall -> (other VMs, but not 'work')

     sys-wifi -> work

Should the redirection stay active via sys-wifi? I think it should not,
at least not automatically (maybe have an option for that?).

I understand all of your points and consequently it is hard to figure out a catch-all solution.

I tried charting the flow of the possible solution.
https://git.lsd.cat/Qubes/gsoc/src/master/assets/implementation.png

As a sum up:
1) Rules are stored only in <appvm>/firewall.xml
2) Rules can either be internal or exteranl (ie: they are applied only to <firewallvm> or both to <firewallvm> and its <netvm>)
3) Forwarding rules should be purged if <appvm> changes <firewall> (maybe also if <firewallvm> changes <netvm>? But that would be harde to detect I guess)
4) Users should be able to specify both the forwarded port and destination port as you were saying
5) Users should be able to eventually restrict forwarding to designated networks (with 0.0.0.0/0 being the wildcard instead of being a wildcard by default)

However, in this case it will surely be harder to display the rules in all the affected vms.
The other approach, as you were suggesting, of adding each specific rule in each vm conf does make sense, but I think then it would necessary something to keep track of the rule dependencies (such as a unique identifier). Furthermore there is a higher risk of having orphaned rules or a inconsistent state.

Furthermore, in the "internal" vpn case that I have in mind, the idea is to forward the local port via the VPN interface or Tor (but in the Tor case users should just stick to Whonix). Some providers, such as Mullvad, AirVPN, PIA etc allows port forwarding this way and I think that's the most relevant case since it allows exposing a service on the internet while maintaining a bit of privacy/anonimity and whithout needing to bypass the local network NAT. Is this the same case you are referring to?

And finally, don't forget IPv6 exists. Which means you can have the same
port on IPv4 and IPv6. And theoretically they could be redirected to
different places (but I'm not sure if that's a good idea...).

I think that once we have figured out the overall logic to implement, it should not be hard to duplicate it for ipv4/ipv6. I think the main problem to think about is to insert proper checks to prevent users from adding mixed rules.

Cheers
Giulio