Those of you who’ve been trying out the new Safari 4 beta – at least on the Mac, though I imagine you could find similar data trails on the Windows version too – have no doubt been impressed at its shiny new features. But if you’re a stickler for disk space like I am, or a stickler for privacy (or, heaven help you, both), Safari’s poor housekeeping is quite alarming.
Let’s start with the easy stuff. In ~/Library/Caches/Metadata/Safari reside two folders: Bookmarks and History. Inside the history folder is a file for every webpage you’ve visited, regardless of when you’ve set Safari to delete history items in your preferences. I suspect Safari does some cleanup here as the files become slightly more sparse as one travels farther back in time (the past week – the time I’ve set Safari to delete history items after – usually comprise about half of the items, and I’m pretty sure my browsing habits aren’t increasing logarithmically), but I cannot figure out what gets kept and what doesn’t. Each item can be anywhere from 4-200k, but those add up when you get thousands of them. Deleting everything past a week old saved me over 100 MB.
This particular dump has been around at least since a version of Safari 3, but Safari 4 is even more egregiously unhygenic. You know the fancy new Top Sites feature, and how it tells you with a little blue star peeled away from the page preview if it’s been updated since you last checked? Safari makes a little file for every site, every time it checks on them, which if RefreshInterval is in seconds as I suspect it is, means it creates a nice XML file for every one of your top sites every 30 minutes (1800 seconds). These are located in ~/Library/PubSub/Feeds/ and given arcane hexadecimal names, and contain whatever turned out to be new on the webpage. As the Wikipedia homepage is one of mine and changes just about constantly, the vast majority of my XMLs were filled with Wikipedia content. I had over 24000, and deleting everything more than a week old (again, about half of the items) saved me about 93 MB.
But even this isn’t the worst of it. The most outrageous thing I found, and it took drinking from Spotlight’s firehose of filesystem changes with FSEventer to find it, was that Safari does not delete the webpage previews it generates for Quicklook. Ever. 2.03 GB of webpage previews (2 per website – a full resolution and a thumbnail), all generated since downloading the Safari 4 beta, residing – not in the user library, not even in the root library – in /private/var/folders/et/etuAKaR1GTeV9DVeRGfst++++TI/-Caches-/com.apple.Safari/Webpage Previews/, a hidden folder far away from the mouseclicks of all but the most relentless tinkerers.
This is completely unacceptable, for two reasons: The first is that there is no reason for Safari not to clean up after itself and let these folders get this big. If it grew to over 2 GB in just a few months, was it just going to grow until I ran out of disk space next year? What of the users who don’t know to delete it? As much as Apple might like to think we have practically infinite disk space for it to throw its crap everywhere, truth is, not all of us can afford terabyte drives to offload our media onto. The average user is going to have no idea where his disk space has gone, and 2 GB is definitely a noticeable amount. I can’t think of a good reason they couldn’t have at least put it in the user library with everything else: hiding it there is nothing short of deceptive.
Secondly, this is a huge privacy concern. With no good way of getting rid of them except manually (clearing the history doesn’t do it) these hidden files are strewn all over the user’s hard drive unbeknownst to him waiting for snooping relatives (or more pertinently, law enforcement) to dig them up.
I really like Safari, but I’m going to have to seriously consider using Firefox now (ack). Or maybe Chrome will arrive with enough polish to take over.
Update: It seems this is indeed a bug and not a feature, possibly confined only to Leopard systems as Tiger users’ histories seem to clear when they’re set to. Hopefully this will be fixed in the final Safari 4. Also fixed miscellaneous typographical errors.
69 Responses
May 23 at 3:28 pm
The “Clear Cache” button clears this information away as cited in the Slashdot link to this article.
http://yro.slashdot.org/article.pl?sid=09/05/23/1747241
May 23 at 3:52 pm
No, it doesn’t. It reduced my Cached folder size but tons of it is still there.
http://yro.slashdot.org/comments.pl?sid=1243323&cid=28069459
May 23 at 3:56 pm
Why worry about Chrome showing up, why worry about switching to Firefox?
Opera is fantastic. Give it a shot!
May 23 at 3:58 pm
That’s why I use Firefox 3…and XP.
I am a PC.
May 23 at 4:47 pm
” … These are located in ~/PubSub/Feeds/ and given arcane hexadecimal names, and contain whatever turned out to be new on the webpage. …”
Umm, no, they’re not. At least, they’re not at ~/PubSub/ …
They are at ~/Library/PubSub/Feeds
The Tilde is a shortcut to the path of the user’s home directory, not the User/Library/ directory, as it’s used in the blog article.
Good post, overall, though.
May 23 at 4:55 pm
Oops, good catch, Johnny. The path is fixed now.
May 23 at 4:56 pm
Well it is a beta, go and tell them how you feel.
May 23 at 4:57 pm
I’ve checked all those directories and while I’ve found the information you are claiming, they were only from my last days of history.
Also, clicking on Safari -> Reset Safari menu item, they were all automatically deleted.
Still, good intentions… so thanks for posting :D
May 23 at 4:58 pm
by the way its not in /private/var/folders/et/ for everyone, but if you get info on the folders in there look out for the several gig one (4.5 gig of webpages for me…)
May 23 at 4:59 pm
Look at “Reset Safari…” in the Safari menu. It will let you clear out all of this stuff.
May 23 at 5:03 pm
@PC: you act like Firefox dosen’t save history to disk and other junk…
This is what browser do. At least Safari has had Reset Safari and Private Browsing for the past 2 versions.
May 23 at 5:06 pm
Good points, but one is flawed. First of all, as pointed out already and fixed, the PubSub folder is in ~/Library. The larger incorrect part: it’s not just Safari that uses it. If you look in Clients.plist, you’ll see all the applications that have used it — it appears to be an RSS feed manager. The feeds listed reflect all those applications, and I’m pretty sure fairly little of it has to do with Top Sites,.
May 23 at 5:07 pm
Overdramatic and paranoid blogger claims Safari’s history files are a giant leak capable of landing innocent users in the gulag and consuming entire harddrives, film at 11.
Yeesh, perspective much?
May 23 at 5:44 pm
@geekboy642: So you think it is a good idea that Safari just dumps several GB of data on your harddrive and does so without the possibility to turn it off?
I had around 3 GB of data screenshots stored in the “Webpage Previews” directory. And it keeps growing.
Beside the obvious loss of harddrive space it will also slow down the computer when using Safari since it will access that directory for each page loaded.
And this does not include the potential privacy problems. I do not like the idea of screenshot of my internet bank being stored in the filesystem that can be accessed if you ever need to hand in your computer for repair.
May 23 at 6:42 pm
Seriously, you can’t afford a 1TB drive? They’re like, what, $150 at the absolute most now? If you can’t be bothered to upgrade now, when upgrading is so cheap it’s insane, then I have little sympathy for you. May your Safari cache grow to 100GB, artard.
May 23 at 8:32 pm
@ Moneybags
I find it hilarious that you don’t understand the concept of keeping your computer clear, or the fact that the person “might” use a laptop which as far, aren’t graced with 1tb hds, so before you make a “throw money at the problem!” remark, stop, think, and don’t
May 23 at 8:35 pm
On my iBook with 10.4.11, I find none of the stuff you’re complaining about: no ~/Library/Caches/Metadata/Safari/History folder, no ~/Library/PubSub, and no /private/var/folders file.
May 23 at 8:39 pm
@arni2000: Did you look at the screenshot for https (eg. secure sites, like your banks should be)? They don’t get one — and private browsing also seems to stop them being taken.
Of course I assume chrome’s (and firefox in future i imagine) full text search of history is actually done using magic, rather than by invading your privacy like safari does.
May 23 at 11:38 pm
If you are worried about hard drive space or privacy it would make sense not to hang these on the shoulders of a four month old beta software.
That said, this appears to be a bug with your particular installation. Just like others mentioned here, my version has a size of less than 500kb for both history and bookmarks. And I’ve been using Safari 4 constantly since release.
May 23 at 11:57 pm
Yep, got the whole whale of a directory under /privat/var/folders. 3gig, 11,000 files since February. And I’ve had top-shites turned off since then.
May 24 at 6:23 am
I think that a lot of the most egregious issues with storage occur only with 10.5. The previews in question are generated for Quicklook, which is not available in 10.4.
Of course it could merely be somewhere else that I don’t know about.
May 24 at 7:14 am
They seriously need to sort this out. Mac OS X is renowned for keeping itself clean and speedy – why not Safari. This is basic Apple! Get it sorted – I don’t want loads of bollocks that I’m no longer using taking up space on my HDD. Even if I have a large disk.
May 24 at 7:51 am
@Mac users you can grab AppleSript here http://scriptbuilders.net/files/safaricleanup0.5.6.1.html or buils your own one ?
May 24 at 8:02 am
I have 6 subfolders under /private/var/folders, and all of them again have further sub folders and finally folders called Cache most of which I don’t have access to (although of coures I could grant me the respective privileges). So I don’t even know their size, although the whole folders folder has some 500 MB. The com.apple.safari folder is some 250 MB in size in my case.
So what are all the other cache folders about? What’s supposed to be stored in private anyway, in particular as it is invisible and approx. 5 GB in size (not counting all the subfolders that I con’t have access to)?
May 24 at 10:25 am
Webpage Previews/ will be deleted when you delete your history. You may not think so, but give it a few minutes… it takes time to delete all those files.
May 24 at 11:57 am
If you’re concerned about privacy, why in gods name would you consider Chrome?
It’s stored OFF site(whereas this is on your own HD) and uniquely identifies each installation. That’s some creepy shit.
May 24 at 12:31 pm
My screenshots folder in /private was a little under 5GB ! The other folders mentioned weren’t nearly big enough for me to care.
I have Safari set to only clear history manually… so I have everything built up over multiple years’ time. Nothing that I did while in Private Browsing mode was saved for me in any of the mentioned folders… and there were no screenshots from secure HTTP servers (like online banking).
And of course the screenshots are stored in the root directory, but the permissions are set so that you would need to be logged as an administrator to see the files. I have a firmware password set so I feel pretty secure… the only way anybody will ever see my screenshots is by opening my iMac and removing the hard drive. And, like I said, screenshots were not taken for the sites I viewed in Private Browsing mode. So the privacy point is moot as far as I can tell.
The disk space is more than I realized it would be, but we all knew when we downloaded Safari 4 that it saved screenshots in the history… it really shouldn’t be TOO surprising.
These files ought definitely to be stored in the user library, not in /private.
Note to author: Don’t confuse GB (gigabytes) with Gb (gigabits). A byte is eight bits. Bytes are usually used to measure storage space or memory; bits are usually used to measure network traffic bandwidth. I dunno why.
May 24 at 12:33 pm
Also, of course, this software is still in beta. Apple appreciates our feedback, I’m sure, and you can’t complain TOO much if you’re a privacy nut or a disk space nut, since beta programs are expected to have issues.
So if you’re running a kiddie porn ring or something, and privacy is hugely important, you should definitely not use beta software.
May 24 at 2:56 pm
I was so surprised! My folder was 4.5 GB :O I would never suspected that!
May 24 at 3:00 pm
I can’t really reproduce this. your 93 MB folder was only about 11 MB for me, and your 2GB folder was 177 MB. That said maybe it’s because I don’t use the visual history thing much, I think it only generates them if you actually use it.
May 24 at 3:13 pm
For any who find it difficult to see your hidden folders, I find OmniDiskSweeper very useful for finding useless large files. It will very quickly let you home in and identify the worst offenders. Showed me the /private/var/folders/… path in a trice.
And, best of all, it’s free!
May 24 at 4:03 pm
Guys, those files are not cleared out because they are from Safari 3.
May 24 at 5:35 pm
If you worry about discspace overall it would be a good thing for you to make sure you run that your mainenence scripts run reguary and your cache gets cleaned now and then.
My advise is just to use Onyx a bit more often.
It cleans all you mentioned here and much more…
May 24 at 10:45 pm
It’s a BETA! Not to mention that hard drives are not 8GBs anymore, but 800GBs. Also your first few numbers were in Mb’s, did you mean MB’s? Or were you too lazy to convert?
May 25 at 1:19 am
I found the bit byte distinction a bit silly, because Ive never heard of sending a partial byte over the internet. It seems like a ploy by data companies to confuse people into thinking their internet speeds are faster then they actually are.
May 25 at 3:01 am
Good point. BTW I have to manually delete PubSub folders to get correct up to date RSS feeds. No problem with the feeds using Firefox under Leopard or Safari 4 beta using Windows 7 (beta) under Parallels. Data kept in /var/folders are definitiely a serious privacy issue.
May 25 at 3:25 am
Run this in terminal :
defaults write com.apple.Safari DebugSafari4IncludeTopSites -bool FALSE
May 25 at 3:59 am
Mine is 2,16 GB. Damn and I almost never use that feature.
Great article. Ignore the fanboys who are unwilling to accept the reality :)
May 25 at 5:30 am
What a stupid article! Can’t believe it.
May 25 at 8:05 am
Using Safari 4 Public Beta (5528.17) on 10.5.7 Build 9J61, I just reset Safari (remove everything except for Cookies and saved form data) and found that:
– The history files in ~/Library/Caches/Metadata/Safari/History/ were still there directly after reset, but they were being deleted, as I could tell from the file count that was going down in the Finder. I had around 6000 files in there (about 100 MB) but there was none left after about 1 or 2 minutes. So if you still have those files after you reset Safari, take another look a couple of minutes later.
– The Webpage Previews in /private/var/folders///-Caches-/com.apple.Safari/ were gone after the reset, as they should be. No problem here. Maybe those people with 2 GB of previews had them left over from a previous version of Safari 4 or the OS that might have been buggy.
However, I found it interesting that there were two files in that same hidden folder (containing the Webpage Previews), namely Cache.db (104.2 MB) and SafeBrowsing.db (29.3 MB). Even repeatedly resetting and restarting Safari did not do anything to these files. I tried removing them and restarting Safari, and I noticed no change, so I deleted them. After a bit of random browsing around, the SafeBrowsing.db is back (4.7 MB) but the Cache.db isn’t. The Cache.db had been last accessed about 2 weeks ago, so I don’t know what’s up with that one.
– The ~/Library/PubSub/ stuff is a different matter. From the documentation at http://developer.apple.com/documentation/InternetWeb/Conceptual/PubSub/overview/overview.html : »The Publication Subscription technology provides an architecture for your application to subscribe feeds. There are three major components to the technology: the Publication Subscription framework (PubSub.framework), the PubSub Agent, and the PubSub Database.« Take note that PubSub is used not only by Safari, but also by Mail and Xcode, possibly more apps.
I, too, had more than 20,000 files lying around in ~/Library/PubSub/Feeds/ (about 170 MB). Shortly after I deleted them, the folder was already being filled up again by the PubSub agent. I took a look at the ~/Library/PubSub/Clients.plist that contains details on what feeds you are subscribed to. I was apparently subscribed to the English and German wikipedia »Recent Changes« pages as well as some kind of recent YouTube videos and PHP documentation update pages, although I positively do not recall ever having started these subscriptions. I removed the subscription entries from the plist, killed the PubSub agent from the command line and restarted Safari. Safari froze for about 3 minutes while rebuilding the ~/Library/PubSub/Database/Database.sqlite3 (which shrank from over 200 MB to just under 1 MB!), but now my ~/Library/PubSub/Feeds/ folder is and hopefully stays empty.
I’d suggest updating to the most recent versions of Safari and OS X, if you haven’t already. Try repeating the above steps and see what happens.
I just thought that if you keep having problems with big heaps of logging and cache files lying around and not being deleted, you might as well have a permissions problem, i. e. some system service might actually be trying to delete those files but can’t because the permissions have been altered. So fixing file permissions from the Disk Utility might help, too.
May 25 at 8:58 am
Sure, my previews directory is 1G, but it only stretches back two weeks. Not sure I think this is such a disaster.
May 25 at 9:59 am
my /private/var/folders was not the problem only 260 mb…
however my ~/Library/PubSub/Feeds was nearly 6 gigs? damn…
May 25 at 11:26 am
Wouldn’t it be fair to change the title to
Safari 4 Beta’s Messy Trail
?
It’s a beta. Subject to change etc.
May 25 at 1:39 pm
You should try Camino. Mozilla based, faster than Safari (Camino 1.6 > Safari 3, Camino 2.0 Beta >Safari 3 and 4beta
May 25 at 1:57 pm
This article is bias! How dare you say negative facts about Apple products?!! FUck you bias author!
I’m buying the $900 24″ apple monitor to make me feel better about life again.
May 26 at 4:29 am
Anyone tried Safari 4 BETA on Windows to see if the problems are as drastic as they are for 10.5? I speculate not, since QuickLook isn’t available on Windows.
–tonza
May 26 at 5:01 am
I’m not a Mac user these days, but I remember reading on Rixstep that Mac OS X has never been efficient in disk-space use. Apparently every Apple program contains megabytes of duplicate files, and I remember back in the Mac OS 9 days how the installer used to dump 120 megabytes of worthless extensions, control panels, contextual menu items, desktop backgrounds etc. That’s when hard disks were 6 gigabytes. Why is anyone really surprised about Safari 4 being a hard disk hog?
May 26 at 5:21 am
Hi again! On Windows 7 RC, Safari 4 BETA dumps files in C:\Users\sysowner\AppData\Local\Apple Computer\Safari, and it has a couple of large files in it: Cache.db @ 5.941 MB, and SafeBrowsing.db @ 18.285 MB, and a large directory called Webpage Previews @ 7.51 MB (at the time of writing).
I have only started running Safari in Windows for a couple of hours (-ish!), but at this rate, I’d hate to see what happens after a couple of weeks of use!
–tonza
May 26 at 8:36 am
Wonder if this was fixed in the new beta that released with 10.5.7? I don’t have anywhere near the size of folders / files that were posted in the article and am a avid web surfer. It seems to be cleared out from the last time i Reset Safari.
May 26 at 9:04 am
For those with large cache files, when did you last restart your Mac? Any files in the cache folders in private/var/folders should be deleted when you restart (if they aren’t being, then that *is a problem* and it sounds like a bug that should be filed). If you haven’t restarted in a long while, then expect to see more files there. Additionally, you will find that you will require either root access (admin isn’t sufficient) to view the content of any of the folders in there, other than the ones created for your own user account so, short of physical access to your drive, this isn’t as bad a security issue as many are making it out to be. Also, it isn’t just Safari that creates content in the private/var/folders location, but most other apps as well. It is a cache repository after all.
Btw, some of the files you are probably finding in the ~/Library/ folders are likely from Safari 3 and not 4, so it isn’t surprising that they are not cleared when you use Safari 4 to try and do it. They are also of similar size to the ones Safari 4 is creating (i.e. up to several hundred MB) so this is nothing new, either.
May 26 at 9:14 am
Quote: “I’m not a Mac user these days, but I remember reading on Rixstep that Mac OS X has never been efficient in disk-space use.”
OS X is very efficient at disk-space use. Most of those duplicate files are from other language packs and it is necessary to actively choose not to install them when you first install the OS, which is something that many users don’t do. These are needed if you want to run the OS and the apps in a language other than your default. Apple can’t predict who will want to do this or not, so they default to installing all of the supported ones.
However, even if you do choose not to install other language packs when you install the OS for the first time, newly downloaded apps from Apple or third parties will invariably contain support for multiple languages, so there is bloat added in that way (unless they off English – or other language – only builds for download, but that raises costs for them so many don’t).
What you have probably read on Rixstep, is that some developers are very sloppy with what they leave in their application packages – it is a bit too easy to leave unnecessary cruft in your package when you build it if you aren’t careful. This isn’t a fault of the OS per se (though, arguably, it is one of XCode), but one of the individual developer(s) who don’t take care to get rid of it before shipping.
Rixstep was also highly critical of the growing number of cache files you now find in OS X since 10.5 (in the private/var/folders location, fwiw) was released, but that is a recent phenomenon, and, iirc, his criticism of this was more to do with of the bifurcation of locations that these types of files are now stored leading to increased complexity, rather than their actual existence.
May 26 at 10:22 pm
My first 2 folders had 8 Mb combined.
I think you’re just watching too must porn.
May 27 at 5:01 am
Just having a look at the history and caches now, I coant see any of the sites I visisted in private mode (banking, work login pages, etc).
Also most of this data is in your home folder. Only admins and you shoud be able to see it. If really worried, use encryption on the home folders.
For general stuff, and especialy fp the kids, the page preview and thmbnails are worth the space they consume, especially for the kids (mine are 6 and 3) it makes it much easer for them tolocate ther favourite sites. Especially as the coverflow mode looks just like iTunes.
May 27 at 3:21 pm
Thanks for your article!
This is a serious bug at all. I could delete 1.6 GB of “screenshots”.
And I just installed Firefox for privacy issues.
Thanks man!
May 28 at 12:22 am
Go to Safari > Reset Safari. It gets rid of all the items you complain about.
May 28 at 11:05 am
The Path for webpage previews (/private/var/folders/et/etuAKaR1GTeV9DVeRGfst++++TI/-Caches-/com.apple.Safari/Webpage Previews/) seems to be different between computers. On my system the path is:
/private/var/folders/GH/GHKRw8W1G6GmJ2nDb5CjTE+++TM/Caches-/com.apple.Safari/Webpage Previews/.
So you have to look into it to find it.
If you want to browse this path in finder use this commands to enable Show All Files in the Finder:
-in the terminal run:
$ defaults write com.apple.Finder AppleShowAllFiles TRUE
$ killall Finder
I’ll be writing a script soon to automate the delete of those files
May 29 at 12:12 pm
@ Sith – Thanks you, I found my folder, 2 gigs even after resetting safari…!
How would such a script look like? I mean, every user has a different path, could you provide some sample code?
Thanks a lot!
May 30 at 3:29 pm
I have an idea to fix this “huge issue”. Stop using the Beta software and wait until the final product is released, then if the issue persists, whine away…
Jun 01 at 6:03 am
@ Sith
Why bother making life complicated? Just press Command-shift-G in the Finder, type in /private/var/folders and then click Go to open it.
The reason why the folders are called different things is because they are caches and ***temporary***. Restart your Mac and they will be/should be deleted automatically.
Jun 03 at 11:05 am
@ Jonathan:
“Restart your Mac and they will be/should be deleted automatically.”
Have you tested your theory? My folder had files in it from last December. I can assure you I have restarted my Mac at least once in that time. The basis of this entire post was that these files aren’t acting like ***temporary*** files and deleting themselves. They are waiting for the user to take action manually.
Jun 08 at 2:49 pm
Now that the final build of Safari 4 is being pushed, does it contain the same problems?
Jun 08 at 3:48 pm
I think it’s fixed now. I actually checked that folder after resetting safari and it’s only 1MB or so. Much much better :)
Jun 09 at 6:05 am
It’s fixed. I didn’t even have Webpage Previews anymore
Jun 18 at 4:56 pm
somebody at macrumors said its not fixed, they just relocated the folder.
Jun 23 at 8:31 pm
I just deleted 15GB (yes that’s with a “G”) worth of Safari history item. I only had Safari 4 for a few days. I suspected it was the culprit because I literally watched the disk space go to zero as I have no other app but Safari running. Thanks so much for the post! (I’m now typing this in Firefox)
Aug 24 at 8:38 pm
Apple Safari takes more resources compared to Opera and Firefox. sometimes it also freezes so i would still stick to Opera.
Aug 27 at 12:12 pm
Of course as long as Safari4 keeps creating previews for any website is visited, some folder in the system is doomed to be stffed up with gigs of files.
The questions now are:
1) If this temp folder has been moved, where is located now?
2) Are those temp preview files not cleaned-up with a common “clear cache” operation? I would like to know about the XML top sites file as well.
3) Is it possible to simple disable both site preview and top site logging anyway?
Thanks in advance for any answer to come.
Sep 03 at 5:18 pm
On Leo using Safari 4.0.3 the Reset option DOES NOT clear the History folder. I had to delete it manually. I had 30 thousand files in it. I also got rid of the web page previews since I don’t use top sites. Here’s how : http://roderick.dk/blog/2009/08/03/disable-webpage-preview-images-in-safari-4-final/
Sep 28 at 3:37 pm
Hopefully Apple will one day see that the Top Sites feature has some serious flaws, even for users that actually like that feature.