👩‍🏫[HowTo] DNS based advertising filter using Bind9

TriMoon · 9 November 2020 13:14

Difficulty: ★★★☆☆
Pre-requisites: Working local Bind9 DNS server.

Steps:

To make your local DNS server become an advertising filter all you need to do is make use of a Response policy zone.
To do that in Bind9 we make use of RPZ via the response-policy section in the options configuration, eg:
```
options {
// RPZ Config
	response-policy {
		zone "redirect"			policy given;
		zone "captive-portal"	policy cname	captive-portal.local.;
		zone "noads"			policy nxdomain;
	};
};
```
- The advertising filtering will be done via the noads zone.
- The other zones i defined above, redirect and captive-portal, can be left out if you don’t make use of them.
  I only added them to show how to make use of multiple RPZs for different purposes.
or use include “/etc/bind/conf/options/rpz.conf”; instead in its place and put the directive in it’s own file for easy administration.
Feel free to use a different file/path for the actual config piece, this path is just what i personally use…
/etc/bind/conf/options/rpz.conf
```
// RPZ Config
response-policy {
	zone "redirect"			policy given;
	zone "captive-portal"	policy cname	captive-portal.local.;
	zone "noads"			policy nxdomain;
};
```

After making that change in the options configuration we ofcourse also need to tell Bind9 where to find the zone file(s).
We do that by defining a similarly named zone(s) at the top level of the configuration, eg.:

/etc/bind/conf/rpz.conf

/**
 * For use in response-policy
 */
 zone "redirect" {
 	type master;
 	file "/etc/bind/zones/rpz/redirect.zone";
 	allow-query { private; corpnets; };
 //	allow-query {none;};
 	zone-statistics full;
 	serial-update-method date;
 };
zone "captive-portal" {
	type master;
	file "/etc/bind/zones/rpz/captive-portal.zone";
	allow-query { private; corpnets; };
//	allow-query {none;};
	zone-statistics full;
	serial-update-method date;
};
zone "noads" {
	type master;
	file "/etc/bind/zones/rpz/noads.zone";
	allow-query { private; corpnets; };
//	allow-query {none;};
	zone-statistics full;
	serial-update-method date;
};

Now the last part to make it complete we create the zone file(s) using the normal syntax of zone file definitions.
Below i will only post the contents of my current “noads.zone” file

/etc/bind/zones/rpz/noads.zone

; -*- Bind9-Zone -*-
; Response Policy Zone (RPZ) file for NOADS
; See: "Response Policy Zone (RPZ) Rewriting" in Bind9-ARM
; http://localhost/doc/bind9-doc/arm/Bv9ARM.ch06.html#id2589969
;
$TTL	1d ; default TTL
@	IN	SOA	ns.lan. root.lan. (
			2020110901	; Serial
			7d		; Refresh
			24h		; Retry
			28d		; Expire
			7d )		; Negative Cache TTL
	IN	NS	ns.lan.
	IN	NS	localhost.

; The domains we want to filter.
; QNAME policy records.  There are no periods (.) after the owner names.

;; google
google-analytics.com			CNAME .		; NXDOMAIN policy
*.google-analytics.com			CNAME .		; NXDOMAIN policy
googleanalytics.com				CNAME .		; NXDOMAIN policy
*.googleanalytics.com			CNAME .		; NXDOMAIN policy
googleadservices.com			CNAME .		; NXDOMAIN policy
*.googleadservices.com			CNAME .		; NXDOMAIN policy
googlesyndication.com			CNAME .		; NXDOMAIN policy
*.googlesyndication.com			CNAME .		; NXDOMAIN policy
;pagead2.googlesyndication.com		CNAME .		; NXDOMAIN policy;; Unneeded because of the wildcards above but included for completeness.
googletagservices.com			CNAME .		; NXDOMAIN policy
*.googletagservices.com			CNAME .		; NXDOMAIN policy
safebrowsing.google.com			CNAME .		; NXDOMAIN policy
safebrowsing-cache.google.com		CNAME .		; NXDOMAIN policy

;; Windows 10
a.ads1.msn.com					CNAME .		; NXDOMAIN policy
a.ads2.msads.net				CNAME .		; NXDOMAIN policy
a.ads2.msn.com					CNAME .		; NXDOMAIN policy
a.rad.msn.com					CNAME .		; NXDOMAIN policy
a-0001.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0002.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0003.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0004.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0005.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0006.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0007.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0008.a-msedge.net				CNAME .		; NXDOMAIN policy
a-0009.a-msedge.net				CNAME .		; NXDOMAIN policy
ac3.msn.com					CNAME .		; NXDOMAIN policy
ad.doubleclick.net				CNAME .		; NXDOMAIN policy
adnexus.net					CNAME .		; NXDOMAIN policy
adnxs.com					CNAME .		; NXDOMAIN policy
ads.msn.com					CNAME .		; NXDOMAIN policy
ads1.msads.net					CNAME .		; NXDOMAIN policy
ads1.msn.com					CNAME .		; NXDOMAIN policy
aidps.atdmt.com					CNAME .		; NXDOMAIN policy
aka-cdn-ns.adtech.de				CNAME .		; NXDOMAIN policy
a-msedge.net					CNAME .		; NXDOMAIN policy
apps.skype.com					CNAME .		; NXDOMAIN policy
az361816.vo.msecdn.net				CNAME .		; NXDOMAIN policy
az512334.vo.msecdn.net				CNAME .		; NXDOMAIN policy
b.ads1.msn.com					CNAME .		; NXDOMAIN policy
b.ads2.msads.net				CNAME .		; NXDOMAIN policy
b.rad.msn.com					CNAME .		; NXDOMAIN policy
bs.serving-sys.com				CNAME .		; NXDOMAIN policy
c.atdmt.com					CNAME .		; NXDOMAIN policy
c.msn.com					CNAME .		; NXDOMAIN policy
cdn.atdmt.com					CNAME .		; NXDOMAIN policy
cds26.ams9.msecn.net				CNAME .		; NXDOMAIN policy
compatexchange.cloudapp.net			CNAME .		; NXDOMAIN policy
corpext.msitadfs.glbdns2.microsoft.com		CNAME .		; NXDOMAIN policy
cs1.wpc.v0cdn.net				CNAME .		; NXDOMAIN policy
db3aqu.atdmt.com				CNAME .		; NXDOMAIN policy
ec.atdmt.com					CNAME .		; NXDOMAIN policy
fe2.update.microsoft.com.akdns.net		CNAME .		; NXDOMAIN policy
feedback.microsoft-hohm.com			CNAME .		; NXDOMAIN policy
flex.msn.com					CNAME .		; NXDOMAIN policy
g.msn.com					CNAME .		; NXDOMAIN policy
h1.msn.com					CNAME .		; NXDOMAIN policy
lb1.www.ms.akadns.net				CNAME .		; NXDOMAIN policy
live.rads.msn.com				CNAME .		; NXDOMAIN policy
m.adnxs.com					CNAME .		; NXDOMAIN policy
m.hotmail.com					CNAME .		; NXDOMAIN policy
msedge.net					CNAME .		; NXDOMAIN policy
msftncsi.com					CNAME .		; NXDOMAIN policy
msnbot-65-55-108-23.search.msn.com		CNAME .		; NXDOMAIN policy
msntest.serving-sys.com				CNAME .		; NXDOMAIN policy
pre.footprintpredict.com			CNAME .		; NXDOMAIN policy
preview.msn.com					CNAME .		; NXDOMAIN policy
pricelist.skype.com				CNAME .		; NXDOMAIN policy
rad.live.com					CNAME .		; NXDOMAIN policy
rad.msn.com					CNAME .		; NXDOMAIN policy
s.gateway.messenger.live.com			CNAME .		; NXDOMAIN policy
s0.2mdn.net					CNAME .		; NXDOMAIN policy
schemas.microsoft.akadns.net			CNAME .		; NXDOMAIN policy
static.2mdn.net					CNAME .		; NXDOMAIN policy
statsfe1.ws.microsoft.com			CNAME .		; NXDOMAIN policy
statsfe2.update.microsoft.com.akadns.net	CNAME .		; NXDOMAIN policy
statsfe2.ws.microsoft.com			CNAME .		; NXDOMAIN policy
survey.watson.microsoft.com			CNAME .		; NXDOMAIN policy
view.atdmt.com					CNAME .		; NXDOMAIN policy
www.msftncsi.com				CNAME .		; NXDOMAIN policy

;; Other wildcard
*.ad-x.co.uk				CNAME .		; NXDOMAIN policy
*.adcolony.com				CNAME .		; NXDOMAIN policy
*.adkmob.com				CNAME .		; NXDOMAIN policy
*.adonline.e-kolay.net			CNAME .		; NXDOMAIN policy
*.ads.anyoption.it			CNAME .		; NXDOMAIN policy
*.ads.mopub.com				CNAME .		; NXDOMAIN policy
*.ads.mp.mydas.mobi			CNAME .		; NXDOMAIN policy
*.ads.yahoo.com				CNAME .		; NXDOMAIN policy
*.ads.yimg.com				CNAME .		; NXDOMAIN policy
*.ads.zynga.com				CNAME .		; NXDOMAIN policy
*.adtilt.com				CNAME .		; NXDOMAIN policy
*.amazon-adsystem.com			CNAME .		; NXDOMAIN policy
*.amplitude.com				CNAME .		; NXDOMAIN policy
*.ashleymadison.com			CNAME .		; NXDOMAIN policy
*.buysellads.com			CNAME .		; NXDOMAIN policy
*.carbonads.com				CNAME .		; NXDOMAIN policy
*.conduit.com				CNAME .		; NXDOMAIN policy
*.dbreklam2.net				CNAME .		; NXDOMAIN policy
*.doubleclick.net			CNAME .		; NXDOMAIN policy
*.exgfnetwork.com			CNAME .		; NXDOMAIN policy
*.fastclick.net				CNAME .		; NXDOMAIN policy
*.gameanalytics.com			CNAME .		; NXDOMAIN policy
*.heyzap.com				CNAME .		; NXDOMAIN policy
*.hotjar.com				CNAME .		; NXDOMAIN policy
*.inmobi.com				CNAME .		; NXDOMAIN policy
*.kontera.com				CNAME .		; NXDOMAIN policy
*.ksmobile.com				CNAME .		; NXDOMAIN policy
*.mobileapptracking.com			CNAME .		; NXDOMAIN policy
*.mobula.sdk.duapps.com			CNAME .		; NXDOMAIN policy
*.onlinewebstat.com			CNAME .		; NXDOMAIN policy
*.otomobilfirsati.com			CNAME .		; NXDOMAIN policy
*.quantserve.com			CNAME .		; NXDOMAIN policy
*.sayyac.com				CNAME .		; NXDOMAIN policy
*.sponsorpay.com			CNAME .		; NXDOMAIN policy
*.startappservice.com			CNAME .		; NXDOMAIN policy
*.tapjoyads.com				CNAME .		; NXDOMAIN policy
*.telemetry.mozilla.org			CNAME .		; NXDOMAIN policy
*.traffichaus.com			CNAME .		; NXDOMAIN policy
*.trafficjunky.net			CNAME .		; NXDOMAIN policy
*.trafficstars.com			CNAME .		; NXDOMAIN policy
*.trovi.com				CNAME .		; NXDOMAIN policy
*.virgul.com				CNAME .		; NXDOMAIN policy

;; Other single domains
app-measurement.com			CNAME .		; NXDOMAIN policy
app.adjust.com				CNAME .		; NXDOMAIN policy; Adjust tracking SDK - see: https://firefox-source-docs.mozilla.org/mobile/android/fennec/adjust.html
analytics.localytics.com		CNAME .		; NXDOMAIN policy
analytics.yahoo.com			CNAME .		; NXDOMAIN policy
beacon.wikia-services.com		CNAME .		; NXDOMAIN policy
cm.ksmobile.com				CNAME .		; NXDOMAIN policy
client.midosoo.com			CNAME .		; NXDOMAIN policy
data.flurry.com				CNAME .		; NXDOMAIN policy
delivery.reklamz.com			CNAME .		; NXDOMAIN policy
e.apsalar.com				CNAME .		; NXDOMAIN policy
hit.clickaider.com			CNAME .		; NXDOMAIN policy
hitbox.com				CNAME .		; NXDOMAIN policy
ingameads.gameloft.com			CNAME .		; NXDOMAIN policy
inmobisdk-a.akamaihd.net		CNAME .		; NXDOMAIN policy
kampanya.qnbfinansbank.com		CNAME .		; NXDOMAIN policy; Advert farm of QNB Finansbank.
live.chartboost.com			CNAME .		; NXDOMAIN policy
marketing-ssl.upsight-api.com		CNAME .		; NXDOMAIN policy
media.admob.com				CNAME .		; NXDOMAIN policy
mobile-collector.newrelic.com		CNAME .		; NXDOMAIN policy
my.mobfox.com				CNAME .		; NXDOMAIN policy
pokazuwka.com				CNAME .		; NXDOMAIN policy
pp.appsflyer.com			CNAME .		; NXDOMAIN policy
propellerads.com			CNAME .		; NXDOMAIN policy
ptreklam.com.tr				CNAME .		; NXDOMAIN policy
ptreklamssp.com.tr			CNAME .		; NXDOMAIN policy
reklam.memurlar.net			CNAME .		; NXDOMAIN policy
; This doesnt work...
;reklam*.com				CNAME .		; NXDOMAIN policy
rubiconproject.com			CNAME .		; NXDOMAIN policy
supersonic.ironbeast.io			CNAME .		; NXDOMAIN policy
supersonicads-a.akamaihd.net		CNAME .		; NXDOMAIN policy
track.appsflyer.com			CNAME .		; NXDOMAIN policy
www.leanplum.com			CNAME .		; NXDOMAIN policy; MMA Mobile Marketing Automation - see: https://firefox-source-docs.mozilla.org/mobile/android/fennec/mma.html

Restart the Bind9 DNS server to have it apply our RPZ: systemctl restart named

Check that it works by querying a domain that you have put in the zonefile, fe:

 trimoon@manjaro  ~  resolvectl query googleanalytics.com
googleanalytics.com: resolve call failed: 'googleanalytics.com' not found
 ✘ trimoon@manjaro  ~  host -v googleanalytics.com
Trying "googleanalytics.com"
Host googleanalytics.com not found: 3(NXDOMAIN)
Received 37 bytes from 127.0.0.53#53 in 0 ms
Received 37 bytes from 127.0.0.53#53 in 0 ms
 ✘ trimoon@manjaro  ~  resolvectl query google.com         
google.com: 2a00:1450:4017:806::200e
            216.58.206.174

-- Information acquired via protocol DNS in 219.1ms.
-- Data is authenticated: no
 trimoon@manjaro  ~ 

Other useful related pages:

The_Quantum_Alpha · 12 November 2020 07:32

Hm, might be a good idea to consider that on a laptop…

There’s some interesting quantum effects that occur when you have over 5 000 000 entries in your hosts file…

TriMoon · 12 November 2020 07:38

uhm?
We are not using the /etc/hosts file…

But if you refer to the policy zone file being too large, then you can always use the include directive of the zone definition sysntax to split your total entries into multiple files
(Or use another editor that has no problem with large files )

The_Quantum_Alpha · 12 November 2020 07:41

I use a custom made AI that detects ads, and blocks them!!

The result may become massive, but it is effective…

Thus, that bind9 DNS setup you showed is definitly something i may try in my free time, never heard of that one!

TriMoon · 12 November 2020 07:57

PS: I see you are using single domain names in that file which can be automated by a single *.us.mixmarket.biz entry in the RPZ

The_Quantum_Alpha · 12 November 2020 07:59

Oh, the AI does it when the similar list reaches over 15 entries, in order to be certain to match up the most possible domain!

It will also analyse the whole hosts list, and will predict future ads domain, and i reached a 98% accuracy!

TriMoon · 12 November 2020 08:00

Well AI’s are always in need of improvements…
(Even humans learn and change daily)

The_Quantum_Alpha · 12 November 2020 08:02

Oh, the beginnings where rough… False positive en mass

chomsky · 12 November 2020 08:47

I used the update-hosts-git package before I switched to pihole on my pi4.