Re: bash oom problem

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
> On 04/10/09 11:17, Sharpe, Sam J wrote:
> > 2009/10/4 psmith<psmith@xxxxxxxxxxxxxxxxx>:
> >    
> >> hi list, whilst trying to run this bash command
> >>
> >> for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done>
> >> wl1
> >>      
> > Consider that for a second...
> >
> > You are trying to generate a list of all possible combinations of an 8
> > character word composed of only uppercase letters - that's 26^8
> > combinations (208 Billion). Each word is 8 bytes long, which I make to
> > be 1670616516608 bytes... or to put it another way, 1.5 TB
> >
> > So, you've got 4 GB of virtual memory and you are trying to fit an
> > 1555GB array into it. Simple mathematics says no. Dumping the
> > arguments before it dies is pointless, because it hasn't even got as
> > far as expanding arguments yet.
> >
> > You need to think of another was to do this and I humbly suggest that
> > Bash should not be high on your list.
> >
> >    
> thanks for the reality check sam :o
> 
> do you have any suggestions what/where i should be looking to get my 
> required output?

As Sam says, your initial attempt is trying to produce all 1.5TB at
once, which is why it blows up, but that's going to happen anyway even
if you program the same thing in C. IOW it's not an issue with the Shell
per se, but with the approach to the problem.

Assuming you actually want all 1.5TB of data (really?) you'd be better
consuming it as it's produced, perhaps via a pipe. Of course that
depends on what you're doing with it, which you haven't said.

poc

-- 
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines

[Index of Archives]     [Current Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux