253 lines
15 KiB
HTML
253 lines
15 KiB
HTML
|
<html>
|
||
|
|
<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
|
||
|
|
<title>Re: GSoC Port Forwarding</title>
|
||
|
|
<link rel="important stylesheet" href="">
|
||
|
|
<style>div.headerdisplayname {font-weight:bold;}
|
||
|
|
</style></head>
|
||
|
|
<body>
|
||
|
|
<table border=0 cellspacing=0 cellpadding=0 width="100%" class="header-part1"><tr><td><div class="headerdisplayname" style="display:inline;">Oggetto: </div>Re: GSoC Port Forwarding</td></tr><tr><td><div class="headerdisplayname" style="display:inline;">Mittente: </div>Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com></td></tr><tr><td><div class="headerdisplayname" style="display:inline;">Data: </div>22/06/2021, 04:43</td></tr></table><table border=0 cellspacing=0 cellpadding=0 width="100%" class="header-part2"><tr><td><div class="headerdisplayname" style="display:inline;">A: </div>Giulio <giulio@gmx.com></td></tr><tr><td><div class="headerdisplayname" style="display:inline;">CC: </div>Frédéric Pierret <frederic.pierret@qubes-os.org></td></tr></table><br>
|
||
|
|
<div class="moz-text-plain" wrap=true graphical-quote=true style="font-family: -moz-fixed; font-size: 14px;" lang="x-unicode"><pre wrap class="moz-quote-pre">
|
||
|
|
Hi, I'm replying to both emails at once:
|
||
|
|
|
||
|
|
On Sun, Jun 20, 2021 at 10:50:04PM +0200, Giulio wrote:
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>Hello,
|
||
|
|
<span class="moz-txt-citetags">> </span>sorry for the late reply.
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span>I read a lot of code and I have to admit that I did not grasp the
|
||
|
|
<span class="moz-txt-citetags">> </span>complexity of the Admin API and networking stack before looking so much
|
||
|
|
<span class="moz-txt-citetags">> </span>into it. I think I've got the overall picture, but it will take a little
|
||
|
|
<span class="moz-txt-citetags">> </span>more to fully be confident moving there.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
(...)
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>core-agent-linux/qubesagent/firewall.py<span class="moz-txt-tag">_</span></span>
|
||
|
|
<span class="moz-txt-citetags">> </span>Is the actual file responsible for running nftables and thus
|
||
|
|
<span class="moz-txt-citetags">> </span>adding/deleting/reloading firewall ruless in the target firewall vm. It
|
||
|
|
<span class="moz-txt-citetags">> </span>also resolves DNS names for domain rules. It is run by Admin API (qubesd).
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
This file is run as part of the qubes-firewall service in relevant VMs
|
||
|
|
(especially sys-firewall).
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>manager/qubesmanager/firewall.py<span class="moz-txt-tag">_</span></span>
|
||
|
|
<span class="moz-txt-citetags">> </span>Contains the code for the "Firewall" tab of the "Qube Manager" window.
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>manager/ui/qubemanager.ui<span class="moz-txt-tag">_</span></span>
|
||
|
|
<span class="moz-txt-citetags">> </span>String and properties for the "Qube Manager" UI.
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span>Questions:
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span>1) Should we both support internal port forwarding and external port
|
||
|
|
<span class="moz-txt-citetags">> </span>forwarding? Such as exposing a port for another domain or exposing a
|
||
|
|
<span class="moz-txt-citetags">> </span>port through the public network interface? I would say yes.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
Yes, I think so. Technically, those two cases should be quite similar.
|
||
|
|
See also the case of sys-vpn much lower in the email.
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>2) Should it be possible to add rules with an 'any' clause (both tcp and
|
||
|
|
<span class="moz-txt-citetags">> </span>udp). I would say no because since port forwarding brings a higher
|
||
|
|
<span class="moz-txt-citetags">> </span>attack surface all rules should be as precise as possible.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
Indeed. Furthermore, most services are either TCP or UDP, very few are
|
||
|
|
both (and for those, it's ok to require two rules).
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>3) Since the expire= feature seems to be already implemented (and
|
||
|
|
<span class="moz-txt-citetags">> </span>limited for the expiring full outgoing access) would it be useful to be
|
||
|
|
<span class="moz-txt-citetags">> </span>implemented in gui and cli for every rule? I would say yes since the
|
||
|
|
<span class="moz-txt-citetags">> </span>admin and agent code seems to be already there. The same goes for the
|
||
|
|
<span class="moz-txt-citetags">> </span>"comment=" field.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
Per-rule expire may be tricky to handle at the GUI level, I have no idea
|
||
|
|
how to make the UI for this not very confusing...
|
||
|
|
But the comment field is definitely useful to use.
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>4) How would you implement the management of forwarding rules in the
|
||
|
|
<span class="moz-txt-citetags">> </span>network providing domain (sys-net)? Shall the user add a rule both in
|
||
|
|
<span class="moz-txt-citetags">> </span>the target domain (ie the one with webserver and another one in sys-net)
|
||
|
|
<span class="moz-txt-citetags">> </span>or should it be fully automatic from the first?
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
From the user point of view, I think it should be automated as much as
|
||
|
|
possible. Like, let the user choose which port in which VM redirect to
|
||
|
|
where. There may be cases when such redirection won't be possible - if
|
||
|
|
there is no network path between the two points.
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>5) Users should be able to set forward rules using domain names and not
|
||
|
|
<span class="moz-txt-citetags">> </span>static ip addresses. In this case, the actual ip addresses of the dst
|
||
|
|
<span class="moz-txt-citetags">> </span>domains should be collected in a similr way as currently DNS are
|
||
|
|
<span class="moz-txt-citetags">> </span>resolved in `/core-agent-linux/qubesagent/firewall.py`, would this be good?
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
But here we are mostly talking about IP addresses of different VMs,
|
||
|
|
right? Those can (and should) be resolved at core-admin side, so the VM
|
||
|
|
applying the rules will have all the IP given. In fact VM may not be
|
||
|
|
able to resolve IP of another VM at all.
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>Proposed XML Syntax:
|
||
|
|
<span class="moz-txt-citetags">> </span><rule>
|
||
|
|
<span class="moz-txt-citetags">> </span> <properties>
|
||
|
|
<span class="moz-txt-citetags">> </span> <property name="action">forward</property>
|
||
|
|
<span class="moz-txt-citetags">> </span> <property name="proto">udp</property>
|
||
|
|
<span class="moz-txt-citetags">> </span> <property name="dstports">443-8080-5555</property>
|
||
|
|
<span class="moz-txt-citetags">> </span> </properties>
|
||
|
|
<span class="moz-txt-citetags">> </span><rule>
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
I don't see an important information here: forward to <span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>where<span class="moz-txt-tag">_</span></span>.
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>Proposed Admin API Syntax:
|
||
|
|
<span class="moz-txt-citetags">> </span>action=forward proto=udp dstports=443-8080-5555 [expire=<unix
|
||
|
|
<span class="moz-txt-citetags">> </span>timestamp>] [comment=random text]
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
Similar here, there needs to be a forward target (IP, and possibly a
|
||
|
|
port)
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>
|
||
|
|
<span class="moz-txt-citetags">> </span>I also plan to document, at least partially, the journey into this.
|
||
|
|
<span class="moz-txt-citetags">> </span>As a last question, I'm curious what is your setup in order to test
|
||
|
|
<span class="moz-txt-citetags">> </span>modifications in the aforementioned repos while developing.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
In practice, I have a separate (physical) computer to run various tests.
|
||
|
|
It can be a system installed on an external disk. But there are also
|
||
|
|
other options:
|
||
|
|
1. Test on the system you are developing on - this may be the easiest
|
||
|
|
option, but is also a risky one (it is easy to break things - may be
|
||
|
|
undesirable if you need that system for other stuff too).
|
||
|
|
2. Test in a virtual machine - it's possible to run Qubes OS inside KVM
|
||
|
|
with nested virtualization enabled (included emulated IOMMU - sadly
|
||
|
|
works only Intel only). This requires non-Qubes host. While it is
|
||
|
|
possible to run Qubes inside Qubes, the setup is quite challenging and
|
||
|
|
fragile, so I don't recommend this path.
|
||
|
|
3. As a last resort, I have a bunch test systems and I can give you
|
||
|
|
access to one of them via SSH. But developing network-related changes,
|
||
|
|
accessing the system through network, may not be the wisest thing to
|
||
|
|
do...
|
||
|
|
|
||
|
|
You can also find some helpful info here:
|
||
|
|
<a class="moz-txt-link-freetext" href="https://www.qubes-os.org/doc/#debugging">https://www.qubes-os.org/doc/#debugging</a>
|
||
|
|
|
||
|
|
On Tue, Jun 22, 2021 at 01:49:15AM +0200, Giulio wrote:
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>Hi,
|
||
|
|
<span class="moz-txt-citetags">> </span>sorry for yesterday long and a bit confusing message. I started writing
|
||
|
|
<span class="moz-txt-citetags">> </span>down my documentation and progress here <a class="moz-txt-link-freetext" href="https://git.lsd.cat/Qubes/gsoc">https://git.lsd.cat/Qubes/gsoc</a>
|
||
|
|
<span class="moz-txt-citetags">> </span>so it should me more readable and easier to follow.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
No worries. Some comments for this page:
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>qvm-firewall <vmname> add action=accept dsthost=1.1.1.1 proto=tcp dstports=80-80 command="cloudflare http test rule" expire=+5000
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
It's "comment", not "command". And also, if you ever need to write Admin
|
||
|
|
API level line manually, the comment field must be last.
|
||
|
|
|
||
|
|
(but on qvm-firewall cmdline it is fine as is)
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>Since in the case of port forwarding the target ip address would always be the <vmname> IP address,
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
This is very true. But there needs to be an information where to forward
|
||
|
|
the traffic to (as noted earlier). Plus, possibly a second set of ports
|
||
|
|
(if you want to redirect to a different port).
|
||
|
|
|
||
|
|
</pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
|
||
|
|
<span class="moz-txt-citetags">> </span>I think my main concern now is the question 4 from the aforementioned
|
||
|
|
<span class="moz-txt-citetags">> </span>email. Shall the rules be automatically implemented in all 3 involved
|
||
|
|
<span class="moz-txt-citetags">> </span>vms? (<netvm,firewallvm,appvm>). I think yes, because otherwise it would
|
||
|
|
<span class="moz-txt-citetags">> </span>be counterintuitive to be a partially manual and partially automatic
|
||
|
|
<span class="moz-txt-citetags">> </span>operation. But since it actually 'automatically' exposes more attack
|
||
|
|
<span class="moz-txt-citetags">> </span>surface, by loosening up all 3 vms network rules, I guess that maybe
|
||
|
|
<span class="moz-txt-citetags">> </span>more reasoning on it would be helpful.
|
||
|
|
</pre></blockquote><pre wrap class="moz-quote-pre">
|
||
|
|
|
||
|
|
Yes, but you need to pass the traffic somehow. The direct connection can
|
||
|
|
be achieved with qvm-connnect-tcp (connecting to the target directly
|
||
|
|
using qrexec, bypassing intermediate VMs), but it has its limits (low
|
||
|
|
performance, TCP only). To keep it as actual IP traffic, you need to
|
||
|
|
change firewall rules at all intermediate VMs too.
|
||
|
|
|
||
|
|
Lets have a specific example: in default setup, redirect TCP port 80
|
||
|
|
from the outside, to 'work' VM port 8080.
|
||
|
|
|
||
|
|
The setup looks like this:
|
||
|
|
|
||
|
|
sys-net -> sys-firewall -> work
|
||
|
|
|
||
|
|
For this, you will need those rules:
|
||
|
|
|
||
|
|
1. In sys-net: forward TCP port 80 to sys-firewall
|
||
|
|
2. In sys-firewall: forward TCP port 80 to work, port 8080
|
||
|
|
3. In work: allow TCP port 8080
|
||
|
|
|
||
|
|
Now is the important design question: how to store those rules? If you
|
||
|
|
store them at all three places separately, it
|
||
|
|
will be easier to apply them at runtime, but it will be harder to
|
||
|
|
correlate them in UI. Plus, if any of them get modified/removed, it may
|
||
|
|
be non-trivial to troubleshoot the issue.
|
||
|
|
The other approach is to store the forward rules only in one place (the
|
||
|
|
target, 'work' in this example? or the source, 'sys-net' here?). This
|
||
|
|
way, it's harder to mess thing up. But when applying the rules (building
|
||
|
|
rule sets for qubes-firewall service in all the involved VMs), you need
|
||
|
|
to check several places.
|
||
|
|
Plus, the UI should clearly show such redirected ports at all involved
|
||
|
|
places, because it does affect system security - it must be easy to spot
|
||
|
|
if any redirects are enabled.
|
||
|
|
|
||
|
|
|
||
|
|
To make things more complex (sorry...), there may be a VPN or other
|
||
|
|
proxy service (Tor?) involved. For example:
|
||
|
|
|
||
|
|
sys-net -> sys-firewall -> sys-vpn -> work
|
||
|
|
|
||
|
|
In such a case, the "external" VM for 'work' is not really sys-net, but
|
||
|
|
rather sys-vpn. And actually you need to be careful to not accidentally
|
||
|
|
bypass VPN either by allowing 'work' to communicate outside of the VPN,
|
||
|
|
or (maybe even worse) systems on the LAN (via sys-net) reach inside VPN.
|
||
|
|
|
||
|
|
This case is not easy to solve, because currently core-admin has no idea
|
||
|
|
whether sys-vpn (or other such VM) do any of such tunnelling. Maybe we
|
||
|
|
need to (finally) introduce some flag to mark such VMs?
|
||
|
|
|
||
|
|
|
||
|
|
And another question: what should happen if you change netvm of 'work'.
|
||
|
|
For example switch to something like:
|
||
|
|
|
||
|
|
sys-net -> sys-firewall -> (other VMs, but not 'work')
|
||
|
|
|
||
|
|
sys-wifi -> work
|
||
|
|
|
||
|
|
Should the redirection stay active via sys-wifi? I think it should not,
|
||
|
|
at least not automatically (maybe have an option for that?).
|
||
|
|
|
||
|
|
|
||
|
|
And finally, don't forget IPv6 exists. Which means you can have the same
|
||
|
|
port on IPv4 and IPv6. And theoretically they could be redirected to
|
||
|
|
different places (but I'm not sure if that's a good idea...).
|
||
|
|
|
||
|
|
<div class="moz-txt-sig">--
|
||
|
|
Best Regards,
|
||
|
|
Marek Marczykowski-Górecki
|
||
|
|
Invisible Things Lab
|
||
|
|
</div></pre></div></body>
|
||
|
|
</html>
|
||
|
|
</table></div>
|