README.md 12.6 KB
Newer Older
Li Mingzhe's avatar
Li Mingzhe committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377
# compressing

[![NPM version][npm-image]][npm-url]
[![build status][travis-image]][travis-url]
[![Test coverage][codecov-image]][codecov-url]
[![David deps][david-image]][david-url]
[![Known Vulnerabilities][snyk-image]][snyk-url]
[![npm download][download-image]][download-url]

[npm-image]: https://img.shields.io/npm/v/compressing.svg?style=flat-square
[npm-url]: https://npmjs.org/package/compressing
[travis-image]: https://img.shields.io/travis/node-modules/compressing.svg?style=flat-square
[travis-url]: https://travis-ci.org/node-modules/compressing
[codecov-image]: https://codecov.io/gh/node-modules/compressing/branch/master/graph/badge.svg
[codecov-url]: https://codecov.io/gh/node-modules/compressing
[david-image]: https://img.shields.io/david/node-modules/compressing.svg?style=flat-square
[david-url]: https://david-dm.org/node-modules/compressing
[snyk-image]: https://snyk.io/test/npm/compressing/badge.svg?style=flat-square
[snyk-url]: https://snyk.io/test/npm/compressing
[download-image]: https://img.shields.io/npm/dm/compressing.svg?style=flat-square
[download-url]: https://npmjs.org/package/compressing

The missing compressing and uncompressing lib for node.

Currently supported:

- tar
- gzip
- tgz
- zip

## Install

```bash
npm install compressing
```

## Usage

### Compress a single file

Use gzip as an example, tar, tgz and zip is same as gzip.

__promise style__

```js
const compressing = require('compressing');

// compress a file
compressing.gzip.compressFile('file/path/to/compress', 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);

// compress a file buffer
compressing.gzip.compressFile(buffer, 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);

// compress a stream
compressing.gzip.compressFile(stream, 'path/to/destination.gz')
.then(compressDone)
.catch(handleError);
```

__stream style__

```js
const compressing = require('compressing');

new compressing.gzip.FileStream({ source: 'file/path/to/compress' })
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.gz'))
  .on('error', handleError);

// It's a transform stream, so you can pipe to it
fs.createReadStream('file/path/to/compress')
  .on('error', handleError)
  .pipe(new compressing.gzip.FileStream())
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.gz'))
  .on('error', handleError);

// You should take care of stream errors in caution, use pump to handle error in one place
const pump = require('pump';)
const sourceStream = fs.createReadStream('file/path/to/compress')
const gzipStream = new compressing.gzip.FileStream();
const destStream = fs.createWriteStream('path/to/destination.gz');
pump(sourceStream, gzipStream, destStream, handleError);
```


### Compress a dir

Use tar as an example, tgz and zip is same as gzip.

__Gzip only support compressing a single file. if you want to compress a dir with gzip, then you may need tgz instead.__

__promise style__

```js
const compressing = require('compressing');
compressing.tar.compressDir('dir/path/to/compress', 'path/to/destination.tar')
.then(compressDone)
.catch(handleError);
```

__stream style__

```js
const compressing = require('compressing');

const tarStream = new compressing.tar.Stream();
tarStream.addEntry('dir/path/to/compress');

tarStream
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/destination.tar'))
  .on('error', handleError);

// You should take care of stream errors in caution, use pump to handle error in one place
const tarStream = new compressing.tar.Stream();
tarStream.addEntry('dir/path/to/compress');
const destStream = fs.createWriteStream('path/to/destination.tar');
pump(tarStream, destStream, handleError);
```

Stream is very powerful, you can compress multiple entries in it;

```js
const tarStream = new compressing.tar.Stream();
// dir
tarStream.addEntry('dir/path/to/compress');

// file
tarStream.addEntry('file/path/to/compress');

// buffer
tarStream.addEntry(buffer);

// stream
tarStream.addEntry(stream);

const destStream = fs.createWriteStream('path/to/destination.tar');
pipe(tarStream, destStream, handleError);
```

### Uncompress a file

__promise style__

```js
const compressing = require('compressing');

// uncompress a file
compressing.tgz.uncompress('file/path/to/uncompress.tgz', 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);

// uncompress a file buffer
compressing.tgz.uncompress(buffer, 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);

// uncompress a stream
compressing.tgz.uncompress(stream, 'path/to/destination/dir')
.then(uncompressDone)
.catch(handleError);
```

**Note: tar, tgz and zip have the same uncompressing API as above: destination should be a path of a directory, while that of gzip is slightly different: destination must be a file or filestream.**


And working with urllib is super easy. Let's download a tgz file and uncompress to a directory:

```js
const urllib = require('urllib');
const targetDir = require('os').tmpdir();
const compressing = require('compressing');

urllib.request('http://registry.npmjs.org/pedding/-/pedding-1.1.0.tgz', {
  streaming: true,
  followRedirect: true,
})
.then(result => compressing.tgz.uncompress(result.res, targetDir))
.then(() => console.log('uncompress done'))
.catch(console.error);
```

__stream style__

```js
const compressing = require('compressing');
const mkdirp = require('mkdirp');

function onEntry(header, stream, next) => {
  stream.on('end', next);

  // header.type => file | directory
  // header.name => path name

  if (header.type === 'file') {
    stream.pipe(fs.createWriteStream(path.join(destDir, header.name)));
  } else { // directory
    mkdirp(path.join(destDir, header.name), err => {
      if (err) return handleError(err);
      stream.resume();
    });
  }
}

new compressing.tgz.UncompressStream({ source: 'file/path/to/uncompress.tgz' })
  .on('error', handleError)
  .on('finish', handleFinish) // uncompressing is done
  .on('entry', onEntry);

// It's a writable stream, so you can pipe to it
fs.createReadStream('file/path/to/uncompress')
  .on('error', handleError)
  .pipe(new compressing.tgz.UncompressStream())
  .on('error', handleError)
  .on('finish', handleFinish) // uncompressing is done
  .on('entry', onEntry);
```

**Note: tar, tgz and zip have the same uncompressing streaming API as above: it's a writable stream, and entries will be emitted while uncompressing one after one another, while that of gzip is slightly different: gzip.UncompressStream is a transform stream, so no `entry` event will be emitted and you can just pipe to another stream**

This constrants is brought by Gzip algorithm itself, it only support compressing one file and uncompress one file.

```js
new compressing.gzip.UncompressStream({ source: 'file/path/to/uncompress.gz' })
  .on('error', handleError)
  .pipe(fs.createWriteStream('path/to/dest/file'))
  .on('error', handleError);
```

## API

### compressFile

Use this API to compress a single file. This is a convenient method, which wraps FileStream API below, but you can handle error in one place.

- gzip.compressFile(source, dest, opts)
- tar.compressFile(source, dest, opts)
- tgz.compressFile(source, dest, opts)
- zip.compressFile(source, dest, opts)

Params

- source {String|Buffer|Stream} - source to be compressed, could be a file path, buffer, or a readable stream
- dest {String|Stream} - compressing destination, could be a file path(eg. `/path/to/xx.tgz`), or a writable stream.
- opts {Object} - usually you don't need it

Returns a promise object.

### compressDir

Use this API to compress a dir. This is a convenient method, which wraps Stream API below, but you can handle error in one place.

__Note: gzip do not have a compressDir method, you may need tgz instead.__

- tar.compressDir(source, dest, opts)
- tgz.compressDir(source, dest, opts)
- zip.compressDir(source, dest, opts)

Params

- source {String|Buffer|Stream} - source to be compressed
- dest {String|Stream} - compressing destination, could be a file path(eg. `/path/to/xx.tgz`), or a writable stream.
- opts {Object} - usually you don't need it

### uncompress

Use this API to uncompress a file. This is a convenient method, which wraps UncompressStream API below, but you can handle error in one place. RECOMMANDED.

- tar.uncompress(source, dest, opts)
- tgz.uncompress(source, dest, opts)
- zip.uncompress(source, dest, opts)
- gzip.uncompress(source, dest, opts)

Params

- source {String|Buffer|Stream} - source to be uncompressed
- dest {String|Stream} - uncompressing destination. When uncompressing tar, tgz and zip, it should be a directory path (eg. `/path/to/xx`). **When uncompressing gzip, it should be a file path or a writable stream.**
- opts {Object} - usually you don't need it
  - opts.zipFileNameEncoding {String} - Only work on zip format, default is 'utf8'.
    Major non-UTF8 encodings by languages:

    - Korean: cp949, euc-kr
    - Japanese: sjis (shift_jis), cp932, euc-jp
    - Chinese: gbk, gb18030, gb2312, cp936, hkscs, big5, cp950

### FileStream

The transform stream to compress a single file.

__Note: If you are not very familiar with streams, just use compressFile() API, error can be handled in one place.__

- new gzip.FileStream(opts)
- new tar.FileStream(opts)
- new tgz.FileStream(opts)
- new zip.FileStream(opts)

Common params:

- opts.source {String|Buffer|Stream} - source to be compressed, could be a file path, buffer, or a readable stream.

Gzip params:

- opts.zlib - {Object} gzip.FileStream uses zlib to compress, pass this param to control the behavior of zlib.

Tar params:

- opts.relativePath {String} - Adds a file from source into the compressed result file as opts.relativePath. Uncompression programs would extract the file from the compressed file as relativePath. If opts.source is a file path, opts.relativePath is optional, otherwise it's required.
- opts.size {Number} - Tar compression requires the size of file in advance. When opts.source is a stream, the size of it cannot be calculated unless load all content of the stream into memory(the default behavior, but loading all data into memory could be a very bad idea). Pass opts.size to avoid loading all data into memory, or a warning will be shown.
- opts.suppressSizeWarning {Boolean} - Pass true to suppress the size warning mentioned.

Tgz params:

tgz.FileStream is a combination of tar.FileStream and gzip.FileStream, so the params are the combination of params of tar and gzip.

Zip params:

- opts.relativePath {String} - Adds a file from source into the compressed result file as opts.relativePath. Uncompression programs would extract the file from the compressed file as relativePath. If opts.source is a file path, opts.relativePath is optional, otherwise it's required.
- opts.yazl {Object} - zip.FileStream compression uses [yazl](https://github.com/thejoshwolfe/yazl), pass this param to control the behavior of yazl.

### Stream

The readable stream to compress anything as you need.

__Note: If you are not very familiar with streams, just use compressFile() and compressDir() API, error can be handled in one place.__

__Gzip only support compressing a single file. So gzip.Stream is not available.__

__Constructor__

- new tar.Stream()
- new tgz.Stream()
- new zip.Stream()

No options in all constructors.

__Instance methods__

- addEntry(entry, opts)

Params

- entry {String|Buffer|Stream} - entry to compress, cound be a file path, a dir path, a buffer, or a stream.
- opts.relativePath {String} - uncompression programs would extract the file from the compressed file as opts.relativePath. If entry is a file path or a dir path, opts.relativePath is optional, otherwise it's required.
- opts.ignoreBase {Boolean} - when entry is a dir path, and opts.ignoreBase is set to true, the compression will contain files relative to the path passed, and not with the path included.

### UncompressStream

The writable stream to uncompress anything as you need.

__Note: If you are not very familiar with streams, just use `uncompress()` API, error can be handled in one place.__

__Gzip only support compressing and uncompressing one single file. So gzip.UncompressStream is a transform stream which is different from others.__

__Constructor__

- new gzip.UncompressStream(opts)
- new tar.UncompressStream(opts)
- new tgz.UncompressStream(opts)
- new zip.UncompressStream(opts)

Common params:

- opts.source {String|Buffer|Stream} - source to be uncompressed, could be a file path, buffer, or a readable stream.

__CAUTION for zip.UncompressStream__

Due to the design of the .zip file format, it's impossible to interpret a .zip file without loading all data into memory.

Although the API is streaming style(try to keep it handy), it still loads all data into memory.

https://github.com/thejoshwolfe/yauzl#no-streaming-unzip-api