I have a personal LAMP webserver running on a RPi2B w/ Raspbian 8 with DDNS service from duckdns and a SSL cert from LetsEncrypt.  

Connecting to the server via HTTP is fine (both local and outside my network), but a few days ago, I noticed HTTPS stopped working, through either duckdns or directly to the IP Address.  I'm not sure what changed.  It might have been after a sudo apt update
All the ports are forwarded correctly (Double NAT) and the cert is current and valid.

apache seems to be in INACTIVE(DEAD) mode, but HTTP still works?  See TERM output below.

seems like apache can't bind to 0.0.0.0:80.  Nothing else that I know of is using that address.  Webmin is running, but on a different port; no torrents.  This speed_test script is running every 30 min, but it shouldn't be blocking port 80 or 443.  EDIT: now that I think about it, it was about the time I loaded the speed_test was the first time I noticed the HTTPS failure, though I had not been on the website in a week or 2.  Not sure if they are related.

Tried rebooting the Pi, sudo service apache2 restart, etc.  Not sure if my Internet Provider started blocking port 443, though all other HTTPS sites work.  I get service through Plateau Fiber.  I doubt it is them, as even on the local network, I cannot connect via HTTPS, only HTTP.

$ service apache2 status

? apache2.service - LSB: Apache2 web server

   Loaded: loaded (/etc/init.d/apache2)

  Drop-In: /lib/systemd/system/apache2.service.d

           +-forking.conf

   Active: inactive (dead) since Sun 2016-10-16 11:47:10 MDT; 5min ago

  Process: 4720 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)

  Process: 4702 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)

apache2/access.log and apache2/error.log are empty
syslog dump after reboot

$ /var/log/syslog

Oct 16 11:47:09 raspberrypi systemd[1]: Starting LSB: Apache2 web server...

Oct 16 11:47:10 raspberrypi apache2[4702]: Starting web server: apache2AH00558: apache2: Could not reliably determine the server's fully 

                                   qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message

Oct 16 11:47:10 raspberrypi apache2[4702]: (98)Address already in use: AH00072: make_sock: could not bind to address [::]:80

Oct 16 11:47:10 raspberrypi apache2[4702]: (98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80

Oct 16 11:47:10 raspberrypi apache2[4702]: no listening sockets available, shutting down

Oct 16 11:47:10 raspberrypi apache2[4702]: AH00015: Unable to open logs

Oct 16 11:47:10 raspberrypi apache2[4702]: Action 'start' failed.

Oct 16 11:47:10 raspberrypi apache2[4702]: The Apache error log may have more information.

Oct 16 11:47:10 raspberrypi apache2[4702]: .

Oct 16 11:47:10 raspberrypi apache2[4720]: Stopping web server: apache2.

Oct 16 11:47:10 raspberrypi systemd[1]: Started LSB: Apache2 web server.

Found this SO post that was very similar (though it seems like his server didn't serve any HTTP or HTTPS):
https://stackoverflow.com/questions/40072846/debian-apache2-doent-start-and-stays-in-inactive-mode

Per his solution, he did not have /usr/sbin/apache2.  I checked /usr/bin/apache2 (note bin vs sbin, d'oh!), which of course didn't come up with anything.  I then reinstalled apache2-bin per his solution, no effect, but I do now have /usr/sbin/apache2  not sure the status of it before the reinstall.

Still can access only via HTTP, not HTTPS.

Search on enabling debug logging.

Netstat -nr. To see if you are listening on 443

From webserver try telnet localhost 443. To see if you can get some kind of connect

Any kind of packet filtering being run on the box?  Get lynx text browser. Try to connect with SSL from the server with that

I'll try that tomorrow, sounds like it might be a bit much for tonight before bed.

pi@raspberrypi:~ $ netstat -nr

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.10.1    0.0.0.0         UG        0 0          0 eth0

10.8.0.0        10.8.0.2        255.255.255.0   UG        0 0          0 tun0

10.8.0.2        0.0.0.0         255.255.255.255 UH        0 0          0 tun0

192.168.10.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0

kyle@mint-laptop ~ $ telnet pi.main 443

Trying 192.168.10.109...

telnet: Unable to connect to remote host: Connection refused

kyle@mint-laptop ~ $ telnet pi.main

Trying 192.168.10.109...

telnet: Unable to connect to remote host: Connection refused

kyle@mint-laptop ~ $ ssh [email protected]

pi@raspberrypi:~ $ telnet localhost 443

Trying ::1...

Trying 127.0.0.1...

telnet: Unable to connect to remote host: Connection refused

pi@raspberrypi:~ $ telnet localhost

Trying ::1...

Trying 127.0.0.1...

telnet: Unable to connect to remote host: Connection refused

Pi-Hole would be the only thing of that type I can think of.  Does that packet filter?  Can't remember if I have connected via https after installing.


From the Pi webserver:
Connects to https://www.google.com just fine.
Does NOT connect to https://localhost  
Does connect to http://localhost

sorry - that netstat -nr should have been "netstat -an"

but that "telnet localhost 443" results you have indicates that either a firewall is on the box blocking things, or you are just not listening of port 443 which is where SSL connections are (due to apache config/misconfig).  the "netstat -an" will tell you which ports you are LISTENing on.

Looks like that was it.  Now to figure out up how to make it listen on 443 again.

pi@raspberrypi:~ $ netstat -an

Active Internet connections (servers and established)

Proto Recv-Q Send-Q Local Address           Foreign Address         State      

tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN     

tcp        0      0 127.0.0.1:587           0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:5901            0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:10000           0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:6001            0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN     

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     

tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN     

tcp        0      0 192.168.10.109:22       172.58.56.98:47100      ESTABLISHED

tcp        0      0 192.168.10.109:22       192.168.10.106:34702    ESTABLISHED

tcp        0      0 192.168.10.109:10000    192.168.10.107:38590    ESTABLISHED

tcp        0      0 192.168.10.109:10000    192.168.10.107:38486    ESTABLISHED

tcp        0      0 192.168.10.109:10000    192.168.10.107:38492    ESTABLISHED

tcp        0      0 192.168.10.109:10000    192.168.10.107:38494    ESTABLISHED

tcp       32      0 192.168.10.109:33046    151.101.53.140:443      CLOSE_WAIT

tcp        0      0 192.168.10.109:10000    192.168.10.107:38484    ESTABLISHED

tcp        0    172 192.168.10.109:22       192.168.10.107:47906    ESTABLISHED

tcp        0      0 192.168.10.109:10000    192.168.10.107:38488    ESTABLISHED

tcp6       0      0 :::80                   :::*                    LISTEN     

tcp6       0      0 :::53                   :::*                    LISTEN     

tcp6       0      0 :::22                   :::*                    LISTEN 

Got it.  It seems lighttpd took over as my host somehow, I remember installing it, but thought I had disabled it.  Forget why I installed it to start.  When I restarted the Pi, lighttpd started first, and was preventing apache from doing it's thing.  Stopped lighttpd, restarted apache2, and now all is well.

I found this post that suggested using ss -tlnp as it gave more detail.  That allowed me to see that lighttpd had grabbed port 80, and since apache was still in a DEAD state, I figured lighttpd might be killing apache.  Sure  enough, it worked.  Now to figure out if I can disable/uninstall lighttpd without further complications.

Thanks for the help, M4!

Since SSL didn't work, you sure lighttpd didn't grab port 443?

 

I think lighttp grabbed 80 (only) first and when apache tried to start, port 80 was already bound, so apache failed then and there, and didn't even try to bind 443.  Therefore, nothing serving 443, only 80.

when I ran ss -tlnp, there was nothing referencing port 443, only lighttpd on port 80