123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253 |
- <html>
- <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
- <title>Re: GSoC Port Forwarding</title>
- <link rel="important stylesheet" href="">
- <style>div.headerdisplayname {font-weight:bold;}
- </style></head>
- <body>
- <table border=0 cellspacing=0 cellpadding=0 width="100%" class="header-part1"><tr><td><div class="headerdisplayname" style="display:inline;">Oggetto: </div>Re: GSoC Port Forwarding</td></tr><tr><td><div class="headerdisplayname" style="display:inline;">Mittente: </div>Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com></td></tr><tr><td><div class="headerdisplayname" style="display:inline;">Data: </div>22/06/2021, 04:43</td></tr></table><table border=0 cellspacing=0 cellpadding=0 width="100%" class="header-part2"><tr><td><div class="headerdisplayname" style="display:inline;">A: </div>Giulio <giulio@gmx.com></td></tr><tr><td><div class="headerdisplayname" style="display:inline;">CC: </div>Frédéric Pierret <frederic.pierret@qubes-os.org></td></tr></table><br>
- <div class="moz-text-plain" wrap=true graphical-quote=true style="font-family: -moz-fixed; font-size: 14px;" lang="x-unicode"><pre wrap class="moz-quote-pre">
- Hi, I'm replying to both emails at once:
- On Sun, Jun 20, 2021 at 10:50:04PM +0200, Giulio wrote:
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>Hello,
- <span class="moz-txt-citetags">> </span>sorry for the late reply.
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span>I read a lot of code and I have to admit that I did not grasp the
- <span class="moz-txt-citetags">> </span>complexity of the Admin API and networking stack before looking so much
- <span class="moz-txt-citetags">> </span>into it. I think I've got the overall picture, but it will take a little
- <span class="moz-txt-citetags">> </span>more to fully be confident moving there.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- (...)
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>core-agent-linux/qubesagent/firewall.py<span class="moz-txt-tag">_</span></span>
- <span class="moz-txt-citetags">> </span>Is the actual file responsible for running nftables and thus
- <span class="moz-txt-citetags">> </span>adding/deleting/reloading firewall ruless in the target firewall vm. It
- <span class="moz-txt-citetags">> </span>also resolves DNS names for domain rules. It is run by Admin API (qubesd).
- </pre></blockquote><pre wrap class="moz-quote-pre">
- This file is run as part of the qubes-firewall service in relevant VMs
- (especially sys-firewall).
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>manager/qubesmanager/firewall.py<span class="moz-txt-tag">_</span></span>
- <span class="moz-txt-citetags">> </span>Contains the code for the "Firewall" tab of the "Qube Manager" window.
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span><span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>manager/ui/qubemanager.ui<span class="moz-txt-tag">_</span></span>
- <span class="moz-txt-citetags">> </span>String and properties for the "Qube Manager" UI.
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span>Questions:
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span>1) Should we both support internal port forwarding and external port
- <span class="moz-txt-citetags">> </span>forwarding? Such as exposing a port for another domain or exposing a
- <span class="moz-txt-citetags">> </span>port through the public network interface? I would say yes.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- Yes, I think so. Technically, those two cases should be quite similar.
- See also the case of sys-vpn much lower in the email.
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>2) Should it be possible to add rules with an 'any' clause (both tcp and
- <span class="moz-txt-citetags">> </span>udp). I would say no because since port forwarding brings a higher
- <span class="moz-txt-citetags">> </span>attack surface all rules should be as precise as possible.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- Indeed. Furthermore, most services are either TCP or UDP, very few are
- both (and for those, it's ok to require two rules).
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>3) Since the expire= feature seems to be already implemented (and
- <span class="moz-txt-citetags">> </span>limited for the expiring full outgoing access) would it be useful to be
- <span class="moz-txt-citetags">> </span>implemented in gui and cli for every rule? I would say yes since the
- <span class="moz-txt-citetags">> </span>admin and agent code seems to be already there. The same goes for the
- <span class="moz-txt-citetags">> </span>"comment=" field.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- Per-rule expire may be tricky to handle at the GUI level, I have no idea
- how to make the UI for this not very confusing...
- But the comment field is definitely useful to use.
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>4) How would you implement the management of forwarding rules in the
- <span class="moz-txt-citetags">> </span>network providing domain (sys-net)? Shall the user add a rule both in
- <span class="moz-txt-citetags">> </span>the target domain (ie the one with webserver and another one in sys-net)
- <span class="moz-txt-citetags">> </span>or should it be fully automatic from the first?
- </pre></blockquote><pre wrap class="moz-quote-pre">
- From the user point of view, I think it should be automated as much as
- possible. Like, let the user choose which port in which VM redirect to
- where. There may be cases when such redirection won't be possible - if
- there is no network path between the two points.
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>5) Users should be able to set forward rules using domain names and not
- <span class="moz-txt-citetags">> </span>static ip addresses. In this case, the actual ip addresses of the dst
- <span class="moz-txt-citetags">> </span>domains should be collected in a similr way as currently DNS are
- <span class="moz-txt-citetags">> </span>resolved in `/core-agent-linux/qubesagent/firewall.py`, would this be good?
- </pre></blockquote><pre wrap class="moz-quote-pre">
- But here we are mostly talking about IP addresses of different VMs,
- right? Those can (and should) be resolved at core-admin side, so the VM
- applying the rules will have all the IP given. In fact VM may not be
- able to resolve IP of another VM at all.
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>Proposed XML Syntax:
- <span class="moz-txt-citetags">> </span><rule>
- <span class="moz-txt-citetags">> </span> <properties>
- <span class="moz-txt-citetags">> </span> <property name="action">forward</property>
- <span class="moz-txt-citetags">> </span> <property name="proto">udp</property>
- <span class="moz-txt-citetags">> </span> <property name="dstports">443-8080-5555</property>
- <span class="moz-txt-citetags">> </span> </properties>
- <span class="moz-txt-citetags">> </span><rule>
- </pre></blockquote><pre wrap class="moz-quote-pre">
- I don't see an important information here: forward to <span class="moz-txt-underscore"><span class="moz-txt-tag">_</span>where<span class="moz-txt-tag">_</span></span>.
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>Proposed Admin API Syntax:
- <span class="moz-txt-citetags">> </span>action=forward proto=udp dstports=443-8080-5555 [expire=<unix
- <span class="moz-txt-citetags">> </span>timestamp>] [comment=random text]
- </pre></blockquote><pre wrap class="moz-quote-pre">
- Similar here, there needs to be a forward target (IP, and possibly a
- port)
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>
- <span class="moz-txt-citetags">> </span>I also plan to document, at least partially, the journey into this.
- <span class="moz-txt-citetags">> </span>As a last question, I'm curious what is your setup in order to test
- <span class="moz-txt-citetags">> </span>modifications in the aforementioned repos while developing.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- In practice, I have a separate (physical) computer to run various tests.
- It can be a system installed on an external disk. But there are also
- other options:
- 1. Test on the system you are developing on - this may be the easiest
- option, but is also a risky one (it is easy to break things - may be
- undesirable if you need that system for other stuff too).
- 2. Test in a virtual machine - it's possible to run Qubes OS inside KVM
- with nested virtualization enabled (included emulated IOMMU - sadly
- works only Intel only). This requires non-Qubes host. While it is
- possible to run Qubes inside Qubes, the setup is quite challenging and
- fragile, so I don't recommend this path.
- 3. As a last resort, I have a bunch test systems and I can give you
- access to one of them via SSH. But developing network-related changes,
- accessing the system through network, may not be the wisest thing to
- do...
- You can also find some helpful info here:
- <a class="moz-txt-link-freetext" href="https://www.qubes-os.org/doc/#debugging">https://www.qubes-os.org/doc/#debugging</a>
- On Tue, Jun 22, 2021 at 01:49:15AM +0200, Giulio wrote:
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>Hi,
- <span class="moz-txt-citetags">> </span>sorry for yesterday long and a bit confusing message. I started writing
- <span class="moz-txt-citetags">> </span>down my documentation and progress here <a class="moz-txt-link-freetext" href="https://git.lsd.cat/Qubes/gsoc">https://git.lsd.cat/Qubes/gsoc</a>
- <span class="moz-txt-citetags">> </span>so it should me more readable and easier to follow.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- No worries. Some comments for this page:
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>qvm-firewall <vmname> add action=accept dsthost=1.1.1.1 proto=tcp dstports=80-80 command="cloudflare http test rule" expire=+5000
- </pre></blockquote><pre wrap class="moz-quote-pre">
- It's "comment", not "command". And also, if you ever need to write Admin
- API level line manually, the comment field must be last.
- (but on qvm-firewall cmdline it is fine as is)
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>Since in the case of port forwarding the target ip address would always be the <vmname> IP address,
- </pre></blockquote><pre wrap class="moz-quote-pre">
- This is very true. But there needs to be an information where to forward
- the traffic to (as noted earlier). Plus, possibly a second set of ports
- (if you want to redirect to a different port).
- </pre><blockquote type=cite style="color: #007cff;"><pre wrap class="moz-quote-pre">
- <span class="moz-txt-citetags">> </span>I think my main concern now is the question 4 from the aforementioned
- <span class="moz-txt-citetags">> </span>email. Shall the rules be automatically implemented in all 3 involved
- <span class="moz-txt-citetags">> </span>vms? (<netvm,firewallvm,appvm>). I think yes, because otherwise it would
- <span class="moz-txt-citetags">> </span>be counterintuitive to be a partially manual and partially automatic
- <span class="moz-txt-citetags">> </span>operation. But since it actually 'automatically' exposes more attack
- <span class="moz-txt-citetags">> </span>surface, by loosening up all 3 vms network rules, I guess that maybe
- <span class="moz-txt-citetags">> </span>more reasoning on it would be helpful.
- </pre></blockquote><pre wrap class="moz-quote-pre">
- Yes, but you need to pass the traffic somehow. The direct connection can
- be achieved with qvm-connnect-tcp (connecting to the target directly
- using qrexec, bypassing intermediate VMs), but it has its limits (low
- performance, TCP only). To keep it as actual IP traffic, you need to
- change firewall rules at all intermediate VMs too.
- Lets have a specific example: in default setup, redirect TCP port 80
- from the outside, to 'work' VM port 8080.
- The setup looks like this:
-
- sys-net -> sys-firewall -> work
- For this, you will need those rules:
- 1. In sys-net: forward TCP port 80 to sys-firewall
- 2. In sys-firewall: forward TCP port 80 to work, port 8080
- 3. In work: allow TCP port 8080
- Now is the important design question: how to store those rules? If you
- store them at all three places separately, it
- will be easier to apply them at runtime, but it will be harder to
- correlate them in UI. Plus, if any of them get modified/removed, it may
- be non-trivial to troubleshoot the issue.
- The other approach is to store the forward rules only in one place (the
- target, 'work' in this example? or the source, 'sys-net' here?). This
- way, it's harder to mess thing up. But when applying the rules (building
- rule sets for qubes-firewall service in all the involved VMs), you need
- to check several places.
- Plus, the UI should clearly show such redirected ports at all involved
- places, because it does affect system security - it must be easy to spot
- if any redirects are enabled.
- To make things more complex (sorry...), there may be a VPN or other
- proxy service (Tor?) involved. For example:
- sys-net -> sys-firewall -> sys-vpn -> work
- In such a case, the "external" VM for 'work' is not really sys-net, but
- rather sys-vpn. And actually you need to be careful to not accidentally
- bypass VPN either by allowing 'work' to communicate outside of the VPN,
- or (maybe even worse) systems on the LAN (via sys-net) reach inside VPN.
- This case is not easy to solve, because currently core-admin has no idea
- whether sys-vpn (or other such VM) do any of such tunnelling. Maybe we
- need to (finally) introduce some flag to mark such VMs?
- And another question: what should happen if you change netvm of 'work'.
- For example switch to something like:
- sys-net -> sys-firewall -> (other VMs, but not 'work')
- sys-wifi -> work
- Should the redirection stay active via sys-wifi? I think it should not,
- at least not automatically (maybe have an option for that?).
- And finally, don't forget IPv6 exists. Which means you can have the same
- port on IPv4 and IPv6. And theoretically they could be redirected to
- different places (but I'm not sure if that's a good idea...).
- <div class="moz-txt-sig">--
- Best Regards,
- Marek Marczykowski-Górecki
- Invisible Things Lab
- </div></pre></div></body>
- </html>
- </table></div>
|