Compare commits

...

1860 Commits

Author SHA1 Message Date
Minio Trusted
b9f622824a Merge master for release. 2017-06-14 12:04:15 -07:00
Harshavardhana
11c4223f2c Add docker release files. (#4473)
We used to release by building directly on the docker
hub auto build process, which is sufficient for edge
but it is not a good idea to do it for stable releases.

Do not build docker release binaries again, but instead
use the released binaries themselves which are signed
and validated.
2017-06-14 02:08:35 -07:00
Nitish Tiwari
26903da8c2 Add steps to remove Minio volumes in the swarm (#4536) 2017-06-13 16:10:11 -07:00
Harshavardhana
353f2d3a6e fs: Hold format.json readLock ref to avoid GC. (#4532)
Looks like if we follow pattern such as

```
_ = rlk
```

Go can potentially kick in GC and close the fd when
the reference is lost, only speculation is that
the cause here is `SetFinalizer` which is set on
`os.close()` internally in `os` stdlib.

This is unexpected and unsual endeavour for Go, but
we have to make sure the reference is never lost
and always dies with the server.

Fixes #4530
2017-06-13 08:29:07 -07:00
Nitish Tiwari
c8947af227 Update Kubernetes-yaml deployment example and Helm deployment doc with Minio image update steps (#4515) 2017-06-13 01:37:14 -07:00
Harshavardhana
075b8903d7 fs: Add safe locking semantics for format.json (#4523)
This patch also reverts previous changes which were
merged for migration to the newer disk format. We will
be bringing these changes in subsequent releases. But
we wish to add protection in this release such that
future release migrations are protected.

Revert "fs: Migration should handle bucketConfigs as regular objects. (#4482)"
This reverts commit 976870a391.

Revert "fs: Migrate object metadata to objects directory. (#4195)"
This reverts commit 76f4f20609.
2017-06-12 17:40:28 -07:00
Harshavardhana
b8463a738c Add support for DCOS host detection, improve Docker detection. (#4525)
isDocker was currently reading from `/proc/cgroup` file. But
this file alone is rather not conclusive evidence. Docker
internally has `.dockerenv` as a special file which we should
use instead.

Fixes #4456
2017-06-13 00:33:21 +00:00
Frank Wessels
6f4862659f Investigate issue #4461 (#4521)
* Code to investigate issue #4461 (rare test failure in TestListenAndServeTLS)

* Use UTCNow() instead of time.Now().UTC()
2017-06-13 00:20:29 +00:00
Nitish Tiwari
f59d7a04b4 Removed references to docker-machine in Swarm guide (#4502) 2017-06-10 21:44:20 -07:00
Dee Koder
b28d5fa633 Clarify macOS instructions on brew paths. Deleted homebrew upgrade instructions. (#4501) 2017-06-09 14:24:32 -07:00
Harshavardhana
48dbd49980 Add support for kubernetes host detection (#4514)
Additionally improve what we print for `docker pull`
such that its precisely the relevant release tag.

Fixes #4456
2017-06-09 02:42:12 -07:00
Bala FA
3dfe254a11 gateway: make each backend as subcommands. (#4506)
Fixes #4450
2017-06-08 23:28:45 -07:00
Krishna Srinivas
ec2920e981 Allow "minio server ." to start minio in fs mode (#4513) 2017-06-08 18:58:51 -07:00
Rushan
0f5483f497 browser: Disable usage/free stats for browser-gateway (#4497) 2017-06-08 15:09:50 -07:00
Krishnan Parthasarathi
8a6b0cc0cd TestInitListeners: Use port 0 pick available port (#4508) 2017-06-08 12:08:21 -07:00
Krishna Srinivas
2c56788f8d Validate gateway arguments (#4376)
Fixes #4355
2017-06-08 11:20:56 -07:00
Frank Wessels
145328ac9f tests: Run select statement in separate goroutine (#4499)
Instead of after the wg.Wait() so as to make sure that the 'earliest'
of the two select case that becomes active is selected..
2017-06-08 07:39:50 -07:00
poornas
45a568dd85 Give more specific error message on browser for nested policies (#4488) 2017-06-07 19:31:23 -07:00
Frank Wessels
7dcc1e92b4 Prevent unnecessary (superfluous) initialization of return variable (#4490) 2017-06-08 00:24:46 +00:00
poornas
999ae1cb96 Fix browser download returning zero bytes for s3 (#4483) 2017-06-06 18:19:35 -07:00
poornas
6651c2fc5f disable settings change on browser in gateway mode (#4472) 2017-06-06 14:56:41 -07:00
Harshavardhana
976870a391 fs: Migration should handle bucketConfigs as regular objects. (#4482)
Current code failed to anticipate the existence of files
which could have been created to corrupt the namespace such
as `policy.json` file created at the bucket top level.

In the current release creating such as file conflicts
with the namespace for future bucket policy operations.
We implemented migration of backend format to avoid situations
such as these.

This PR handles this situation, makes sure that the
erroneous files should have been moved properly.

Fixes #4478
2017-06-06 12:15:35 -07:00
Harshavardhana
f99987e47c Generate sha1sum as well for release for backward compatibility. (#4475)
Additionally remove support for arm6vl in release, since
go 1.8 the support for armv6 has been dropped and we do
not see high usage events from this platform.
2017-06-06 11:25:06 -07:00
Harshavardhana
1c3f244fc5 creds: Secretkey should be generated upto 40 characters in length. (#4471)
Current code allowed it wrongly to generate secret key upto 100
we should only use 100 as a value to validate but for generating
it should be 40.

Fixes #4470
2017-06-05 15:18:03 -07:00
Aditya Manthramurthy
986aa8fabf Bypass network in lock requests to local server (#4465)
This makes lock RPCs similar to other RPCs where requests to the local
server bypass the network. Requests to the local lock-subsystem may
bypass the network layer and directly access the locking
data-structures.

This incidentally fixes #4451.
2017-06-05 12:25:04 -07:00
poornas
2559614bfd fix: Set UIversion in reply for policy API (#4469) 2017-06-05 08:11:54 -07:00
Harshavardhana
a4d1ef1b62 browser: update ui-assets with new changes. (#4467)
Fixes #4269
2017-06-02 15:11:47 -07:00
Harshavardhana
432bf7d99e Fail if formatting is wrong in our CI tests. (#4459)
We didn't fail before, we should helps in avoiding
formatting issues to creep into the codebase.
2017-06-02 14:05:51 -07:00
poornas
18c4e5d357 Enable browser support for gateway (#4425) 2017-06-01 09:43:20 -07:00
Aditya Manthramurthy
64f4dbc272 Disable redirect of HTTP request to a HTTPS Minio server (#4454)
Fixes #4452
2017-05-31 20:33:13 -07:00
Frank Wessels
9ba57a8df0 Add errCorruptedFormat to list of ignored errors for metadata operations. (#4447)
Fixes listing of objects where xl.json is empty or corrupted to skip to the next disk/server (issue 4354).
2017-05-31 20:03:32 -07:00
Dee Koder
5621e6a494 Refactor service stop signal message. (#4428) 2017-05-31 11:53:04 -07:00
Frank Wessels
0f0758aece Load IO error count for posix atomically (#4448)
* Load error count atomically in order to check for maximum allowed number of IO errors.

* Remove unused (previously atomic) network IO error count
2017-05-31 09:22:53 -07:00
Aditya Manthramurthy
a0e02f43e1 Fix and cleanup update message and improve related tests (#4361)
Fixes #4232
2017-05-31 09:22:00 -07:00
Harshavardhana
28352f3f5d log: Startup banner should strip standard ports. (#4443)
APIEndpoints list should strip off standard ports
to avoid confusion with clients.
2017-05-31 09:21:28 -07:00
Harshavardhana
975972d57e server: Redirection should use globalMinioPort with host without port. (#4445)
Currently redirection doesn't work in following scenarios

 - server started with port ":80" and TLS is configured
   client requested insecure request on port "80"
   gets redirected to port 443 and fails.
2017-05-31 09:21:02 -07:00
Harshavardhana
458f22f37c log: Fix printing of signature error request headers. (#4444)
The following commit f44f2e341c
fix was incomplete and we still had presigned URLs printing
in query strings in wrong fashion.

This PR fixes this properly. Avoid double encoding
percent encoded strings such as

`s3%!!(MISSING)A(MISSING)`

Print properly as json encoded.

`s3%3AObjectCreated%3A%2A`
2017-05-31 00:11:06 -07:00
Krishna Srinivas
0bba3cc8e3 gateway-azure: Convert S3 metadata to azure metadata (#4384)
fixes #4292
2017-05-30 20:05:41 -07:00
Cesar Alvernaz
bac5303b10 Some minor fixes (#4441)
Word terminations
2017-05-30 11:03:25 -07:00
Harshavardhana
e01b2fc06d Disable network share test, appveyor bug. (#4446) 2017-05-30 11:02:31 -07:00
Harshavardhana
072fcf3ba6 fs: Make sure to validate bucket first in PutObject() (#4427)
Currently even when bucket doesn't exist we wrongly
return success, when an object is a directory prefix with
 '/' as suffix and is of size 0.

This PR fixes this behavior.
2017-05-25 09:22:43 -07:00
Harshavardhana
b78f6fbcc5 Do not send envVars in ServerInfo() (#4422)
Sending envVars along with access and secret
exposes the entire minio server's sensitive
information. This will be an unexpected
situation for all users.

If at all we need to look for things like if
credentials are set through env, we should
only have access to only this information
not the entire set of system envs.
2017-05-24 21:09:23 -07:00
samkevich
99ca8a2928 fix InvalidAccessKeyId error according to amazon documentation (#4404)
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html
2017-05-23 20:07:52 -07:00
poornas
9b3dd44607 Add dotnet library to minio startup message (#4410) 2017-05-23 13:57:27 -07:00
Krishnan Parthasarathi
3c5db69ffd Treat 0.0.0.0 as local address in --address flag (#4386) 2017-05-23 12:07:39 -07:00
Krishna Srinivas
28c26a9e59 Generate random ETag if client does not provide MD5 for PutObjectPart (#4385)
fixes #4289
fixes #4290
2017-05-22 15:48:48 -07:00
morph027
9136734c02 added secrets to distributed swarm minio (#4374) 2017-05-22 11:30:30 -07:00
Harshavardhana
348aa6566c Add nsswitch.conf to our docker image (#4379)
This will cause the alpine images to resolve DNS
using /etc/hosts along with the normal DNS resolver.
2017-05-22 07:05:39 -07:00
luomeiqin
9d98bf1c0f Bucket names can contain hyphen (#4324) 2017-05-19 07:30:00 -07:00
Rushan
a767ad321a Browser: Fix Safari Blob download issue (#4357) 2017-05-17 17:03:44 -07:00
Harshavardhana
1886d94e95 server/mux: Use constants provided by Go http (#4360) 2017-05-17 11:57:52 -07:00
Anis Elleuch
542f7ae42c gateway: Reject endpoint pointing to local gateway (#4310)
Show an error when the user enters an endpoint url pointing
to the gateway server itself.
2017-05-16 21:13:29 -07:00
Harshavardhana
59b3e0b79b auth/rpc: Add RWMutex instead of Mutex for granular locking. (#4352)
Refer https://github.com/minio/minio/issues/4345
2017-05-16 14:34:56 -07:00
Aditya Manthramurthy
8975da4e84 Add new ReadFileWithVerify storage-layer API (#4349)
This is an enhancement to the XL/distributed-XL mode. FS mode is
unaffected.

The ReadFileWithVerify storage-layer call is similar to ReadFile with
the additional functionality of performing bit-rot checking. It
accepts additional parameters for a hashing algorithm to use and the
expected hex-encoded hash string.

This patch provides significant performance improvement because:

1. combines the step of reading the file (during
erasure-decoding/reconstruction) with bit-rot verification;

2. limits the number of file-reads; and

3. avoids transferring the file over the network for bit-rot
verification.

ReadFile API is implemented as ReadFileWithVerify with empty hashing
arguments.

Credits to AB and Harsha for the algorithmic improvement.

Fixes #4236.
2017-05-16 14:21:52 -07:00
Frank
cae4683971 Make clearing of stale debug lock info independent of deleting map entry of lock itself. (#4353)
This is believed to address issue #4337 where stale information for debug locks in shown.
2017-05-16 07:19:17 -07:00
Krishna Srinivas
5db1e9f3dd signature: use region from Auth header if server's region not configured (#4329) 2017-05-15 18:17:02 -07:00
Anis Elleuch
465274cd21 server-info: Change Error type to string (#4346)
Golang std error type doesn't marshal/unmarshal with json. So errors
are not actually being sent when a client calls ServerInfo() API.
2017-05-15 07:28:47 -07:00
Harshavardhana
87fb911d38 Rename structs for azure and s3 gateway to be consistent. (#4347) 2017-05-15 00:52:33 -07:00
Harshavardhana
155a90403a fs/erasure: Rename meta 'md5Sum' as 'etag'. (#4319)
This PR also does backend format change to 1.0.1
from 1.0.0.  Backward compatible changes are still
kept to read the 'md5Sum' key. But all new objects
will be stored with the same details under 'etag'.

Fixes #4312
2017-05-14 12:05:51 -07:00
Harshavardhana
c63afabc9b build/release: Generate sha256sums also without the release tag. (#4318)
Ref #4306
2017-05-12 21:40:22 -07:00
poornas
0404e747cf Fix broken link (#4344) 2017-05-12 18:29:50 -07:00
Aditya Manthramurthy
3bc9e6101c Add minimum requirements sections to notifications docs (#4328) 2017-05-11 17:34:27 -07:00
Anis Elleuch
f2ed149714 Add slack channel link to corrupted disk err msg (#4270) 2017-05-11 14:27:32 -07:00
Harshavardhana
5a16dcf4cf Add a graceful msg when CTRL+C is pressed. (#4248) 2017-05-11 14:27:18 -07:00
Harshavardhana
fa3d5d0f46 build/release: Generate sha256sums for built binaries. (#4311)
We used to build sha1sum deprecate it and
use sha256sum instead.

Fixes #4306
2017-05-10 11:22:05 -07:00
Krishna Srinivas
bb292e4e38 web-handler: Allow anonymous download of zip (#4309)
fixes #4230
2017-05-10 09:54:24 -07:00
poornas
d1971b9a4d Prevent duplicate policy rows from being created (#4276) 2017-05-10 09:52:31 -07:00
Harshavardhana
fa3f6d75b6 fs: Verify if parent is an object before i/o. (#4304)
PutObject() needs to verify and fail.

Fixes #4301
2017-05-09 17:46:46 -07:00
Harshavardhana
298b470f69 fs/erasure: Ignore objects with / even for DeleteObject() (#4303)
Additionally GetObject() also returns errFileNotFound similar
to HeadObject().

Fixes #4302
2017-05-09 14:32:24 -07:00
Krishna Srinivas
fc774957fe gateway: reject requests with unknown authorization (#4297) 2017-05-09 07:53:31 -07:00
Nitish Tiwari
c6258f5e97 Multi tenancy doc (#4215)
* Add multi-tenancy doc

* Multi-tenancy documents

* Remove intro

* Update deploy-multiple-minio.md

* Update deploy-multiple-minio.md

* Update deploy-multiple-minio.md

* Update deploy-multiple-minio.md

* Update multi-tenant details

* Remove file

* Rename deploy-multiple-minio.md to README.md

* update ports

* Add multi-tenancy diagrams

* Link diagrams and update disk name in the commands

* Fix tenant config directory
2017-05-08 19:22:34 -07:00
Anis Elleuch
85bc6003e9 gateway-s3: Avoid x2 double quotes in ListParts (#4295)
ListParts response returns doubled double quotes in ETag field.
This commit cleans ETag when receiving it from minio client to
fix the issue.
2017-05-08 14:42:05 -07:00
Nitish Tiwari
0d9de50e21 Bump Docker compose file to latest release (#4271)
* Bump Docker compose file to latest release
* Bump Docker Swarm compose file to latest release
2017-05-07 18:58:24 -07:00
Rushan
d13aa1c42d browser: make input number types readonly in share objects modal (#4273) 2017-05-07 18:02:19 -07:00
Harshavardhana
610dbe3479 config: Do not migrate config file if not needed. (#4264)
Also improve the error message returned by `pkg/quick`.

Fixes #4233
2017-05-06 10:16:59 -07:00
Anis Elleuch
2df1e2e9a9 doc: Fix pgsql cmd example (#4265) 2017-05-05 15:20:29 -07:00
Harshavardhana
e372b5ed67 build: Fix release build names. (#4263)
Currently due to the occurrence of 6 arguments from
`gen-ldflags.go` leads to a bug where the binaries
genenerated have wrong names.

As shown below.

```
If you want to build for all, Just press Enter: linux/amd64
-->    linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT]  17MiB minio
[2017-05-04 23:08:51 PDT]  17MiB minio.2017-05-05T06:08:22Z
[2017-05-04 23:08:51 PDT]    76B minio.shasum
```

This PR fixes this issue by retaining the previous release
binary names.

```
If you want to build for all, Just press Enter: linux/amd64
-->    linux/amd64:github.com/minio/minio
$ ls release/linux-amd64/
[2017-05-04 23:08:51 PDT]  17MiB minio
[2017-05-04 23:08:51 PDT]  17MiB minio.RELEASE.2017-05-05T06-08-22Z
[2017-05-04 23:08:51 PDT]    76B minio.shasum
```
2017-05-05 13:16:58 -07:00
Harshavardhana
76f4f20609 fs: Migrate object metadata to objects directory. (#4195)
Fixes #3352
2017-05-05 08:49:09 -07:00
Harshavardhana
99ddd35343 docs: use IEC format such as iB everywhere. (#4247) 2017-05-05 08:28:08 -07:00
Remco Verhoef
01e9adc4b3 Implement anonymous uploads, fixes #4250 (#4259) 2017-05-04 20:03:56 -07:00
Nitish Tiwari
4c63fd06c6 Add Kubernetes yaml file deployment example (#4262)
* Add Kubernetes yaml file deployment example

* Change the Docker image tag to latest

* Update latest release tag
2017-05-04 19:30:46 -07:00
Minio Trusted
4f61bd025d Bump to new release. 2017-05-04 18:31:31 -07:00
Minio Trusted
40985cc4e3 Merge from 'master' 2017-05-04 17:56:58 -07:00
Nitish Tiwari
bb4efbf258 Add minimum requirements for MySQL notification (#4260) 2017-05-04 17:30:56 -07:00
Harshavardhana
a89c7299d1 browser: Update ui-assets with new fixes. (#4246)
Brings two fixes.

 - browser: Listing should append instead of replacing previous listing (#4188)
 - browser: Make login form browser auto-fill compatible (#4091) fixes #4235
 - browser: Selecting a new bucket appends objects list to previous bucket's list (#4252)
2017-05-04 14:57:41 -07:00
Harshavardhana
df027a8f51 Webhook endpoints can fail, we must start the server. (#4255)
This PR fixes a regression introduced in #4060
2017-05-04 13:43:54 -07:00
Aditya Manthramurthy
a02575ebf9 Bump up minio-go to (fixes #4243) (#4256) 2017-05-04 13:43:23 -07:00
Krishna Srinivas
972a527b66 browser: Selecting a new bucket appends objects list to previous bucket's list (#4252) 2017-05-04 11:12:46 -07:00
Krishnan Parthasarathi
02910725c5 Make gateway help for s3/azure similar (#4249) 2017-05-04 10:38:48 -07:00
Harshavardhana
0ea8bfaf78 Add waiting on hosts in docker entrypoint for distributed setups. (#4244)
Thanks to Remco Verhoef <remco@dutchcoders.io> for the script.

Fixes #4225
2017-05-04 00:48:13 -07:00
Remco Verhoef
069cf9e8aa Use s3.amazonaws.com as default endpoint, fixes #4240 (#4242) 2017-05-03 22:41:03 -07:00
Aditya Manthramurthy
2121b78ea7 Fix bug in JSON representation of object properties (#4238)
Introduced in #4003
2017-05-03 20:10:00 -07:00
Remco Verhoef
5016649f47 Add s3 backend to help, fixes #4219 (#4221)
* Add s3 backend to help, fixes #4219

* Add samples for Gateway usage with S3
2017-05-03 17:55:30 -07:00
Bala FA
2b78444056 fix: ignore TLS handshake error. (#4227)
Fixes #4200
2017-05-03 03:23:15 -07:00
Karthic Rao
9b58a669e5 tests: Fix rare test crash (#4175)
Fix rare test crash by improving the randomness logic.
2017-05-02 23:54:22 -07:00
Krishna Srinivas
e5b2e25caf gateway-s3: vendor-update minio-go (#4220) 2017-05-02 18:46:39 -07:00
Krishna Srinivas
4aa65910e5 gateway: Restore bucket policy functionality for Azure (#4209) 2017-05-02 12:27:25 -07:00
Harshavardhana
8b272a3163 config: Improve config migrate messaging. (#4216)
Previous message

```
Migration from version ‘17’ to ‘18’ completed successfully.
```

For example didn't provide any meaningful insights.

This PR attempts to improve this message as below

```
Configuration file '/home/harsha/.minio/config.json' migrated from version '17' to '18' successfully.
```

Fixes #4199
2017-05-02 11:43:27 -07:00
Harshavardhana
f0b5c0ec7c windows: Support all REPARSE_POINT attrib files properly. (#4203)
This change adopts the upstream fix in this regard at
https://go-review.googlesource.com/#/c/41834/ for Minio's
purposes.

Go's current os.Stat() lacks support for lot of strange
windows files such as

 - share symlinks on SMB2
 - symlinks on docker nanoserver
 - de-duplicated files on NTFS de-duplicated volume.

This PR attempts to incorporate the change mentioned here

   https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/

The article suggests to use Windows I/O manager to
dereference the symbolic link.

Fixes #4122
2017-05-02 02:35:27 -07:00
Remco Verhoef
44d53c9c67 cleanup and fix comments (#4212) 2017-05-01 14:44:31 -07:00
Krishna Srinivas
6cf6828a4c gateway: Rename gateway files to have "gateway-" prefix (#4207) 2017-05-01 10:32:18 -07:00
Minio Trusted
1b472dae78 Bump to new release. 2017-04-28 17:58:49 -07:00
Krishna Srinivas
eb50175ad9 gateway: reject bad path segments in URL (#4202) 2017-04-28 17:26:13 -07:00
Krishna Srinivas
e85349381e gateway: Fix help message for gateway (#4201) 2017-04-28 17:26:00 -07:00
Krishna Srinivas
06bc68a4b3 gateway: Fix help message for custom Azure Blob Storage endpoint. (#4113) 2017-04-28 17:23:41 -07:00
Krishna Srinivas
fb506c7fca gateway: Support for custom endpoint. (#4086) 2017-04-28 17:23:23 -07:00
Krishna Srinivas
01f04c717e gateway: reject bad path segments in URL (#4202) 2017-04-28 17:17:18 -07:00
Krishna Srinivas
0d32b22359 gateway: Fix help message for gateway (#4201) 2017-04-28 16:42:16 -07:00
Harshavardhana
cab298d68f pkg: Update the rpm spec with latest release. (#4187) 2017-04-28 12:35:02 -07:00
Krishna Srinivas
1ea53b4d9f browser: Listing should append instead of replacing previous listing (#4188)
Fixes #4144
2017-04-28 09:30:26 -07:00
Anis Elleuch
d36dd80a8a cors: Set Access-Control-Allow-Credentials to true (#4185)
This allow browsers to send credentials with preflighted requests.
2017-04-27 12:40:22 -07:00
Remco Verhoef
3a539ce660 Implement gateway S3 support (#3940) 2017-04-27 11:26:00 -07:00
Harshavardhana
57c5c75611 web: Simplify and converge common functions in web/obj API. (#4179)
RemoveObject() in webAPI currently re-implements some part
of the code to remove objects combine them for simplicity
and code convergence.
2017-04-26 23:27:48 -07:00
Bala FA
cf1fc45142 Improve duration humanization. (#4071) 2017-04-26 03:38:35 -07:00
Dee Koder
64c1c0f37d docs: Update with home brew special note for macOS upgrades. (#4180) 2017-04-25 20:18:48 -07:00
Dee Koder
82857cd6df docs: Document homebrew install path changes for minio. (#4178)
* docs: Document homebrew install path changes for minio.

* updates: Updated with the feedback provided.

* docs: Fix breaking change message as per feedback.

* docs: fix typos in readme.

* typo: grammar refactor of update instructions.
2017-04-25 19:43:22 -07:00
Harshavardhana
8a7cffe7b8 docs: macOS brew now refers to Minio fork (#4059) 2017-04-25 11:01:51 -07:00
Krishnan Parthasarathi
3cdc0c57c8 Provide command to help fill issue template (#4174) 2017-04-25 00:58:11 -07:00
Harshavardhana
dc365bca44 build: -s -w should be added by gen-ldflags.go (#4172) 2017-04-24 23:01:38 -07:00
Harshavardhana
3b1626216d docs: Point docker compose to new release. (#4171) 2017-04-24 21:37:13 -07:00
Minio Trusted
5c85ce1afd Bump to new release. 2017-04-24 18:34:34 -07:00
Harshavardhana
710db6bdad build: Reduce binary size by using -s -w (#4027)
Refer #3939
2017-04-24 18:23:42 -07:00
Harshavardhana
058ea84605 server: Validate path for bad components in a handler. (#4170) 2017-04-24 18:22:40 -07:00
Harshavardhana
48aa2ac392 server: Validate path for bad components in a handler. (#4170) 2017-04-24 18:13:46 -07:00
Frank
0d1e2ab509 Remove hardcoded min and max limit for erasure coding (#4157) 2017-04-24 10:00:33 -07:00
Nitish Tiwari
ebf4c447bb docs: Add Minikube deployment to k8s docs (#4133) 2017-04-24 09:20:51 -07:00
Peter Tribble
2b96d9f706 Enable build on solaris (#4115) 2017-04-23 11:10:18 -07:00
Anis Elleuch
83abad0b37 admin: ServerInfo() returns info for each node (#4150)
ServerInfo() will gather information from all nodes before returning
it back to the client.
2017-04-21 07:15:53 -07:00
Harshavardhana
df346753e1 api: Fix registering of s3 endpoint peers properly (#4159)
We need to have local peer initialized properly
for listen bucket to work, current code did initialize
properly but the resulting code was initializing
peer on a wrong target v/s what listen bucket expected
it to be.

This regression came in de204a0a52

Fixes #4158
2017-04-20 15:28:29 -07:00
Harshavardhana
f1d7780167 lock: Vendorize all the new changes made in minio/dsync (#4154)
Fixes #4139
2017-04-19 14:22:35 -07:00
Harshavardhana
5a3c5aec31 server/mux: Fix serverMux to set deadlines based on UTC time. (#4146)
Avoid using `time.Now()` instead rely on UTC time
for the final deadline, this is to be consistent with
all our internal functions.

Reduce the default read timeout to 15 seconds
in lieu with a newly discovered issue
   - https://github.com/minio/minio/issues/4139

Additionally also change the Read() conn wrapper
to set deadline only upon successful Reads().
2017-04-19 13:16:06 -07:00
Aditya Manthramurthy
a4305742e8 Add key for Kafka messages (fixes #4143) (#4151) 2017-04-19 11:26:35 -07:00
Harshavardhana
640ebb2f79 lock: Fix missing formatting directives while printing. (#4147)
Current log prints in this form

```
ERRO[8150] Lock maintenance failed to remove entry for write
lock (should never happen)%!!(MISSING)(EXTRA ....
```

Fix this by using proper formatting directive.
2017-04-19 10:37:56 -07:00
Harshavardhana
402a5e3bea docs: Fix and reword FreeBSD documentation. (#4145) 2017-04-19 00:25:20 -07:00
Harshavardhana
f4dac979a2 server: Fix message when corrupted or unsupported format is found. (#4142)
Refer https://github.com/minio/minio/issues/4140

This is a fix to provide a little more elaborate message.
2017-04-18 10:35:17 -07:00
Krishnan Parthasarathi
3032f0f505 Remove duration field from lock instrumentation (#4111)
Duration for which a lock was held can be computed from the `Since`
field of `OpsLockState`. It is the difference between current time and
time at which the namespace lock was held. This change avoids
superfluous instrumentation.
2017-04-15 11:40:01 -07:00
Harshavardhana
7765081db7 cache: Increasing caching GC percent from 20 to 50. (#4041)
Previous value was set to avoid large cache value build
up but we can clearly see this can cause lots of GC
pauses which can lead to significant drop in performance.

Change this value to 50% and decrease the value to 25%
once the 75% cache size is used. To have a larger
window for GC pauses.

Another change is to only allow caching if a server has
more than 24GB of RAM instead of 8GB.
2017-04-15 02:16:49 -07:00
Anabel V
18bfe5cba6 docs: Created new illustration for docs. (#4012) 2017-04-14 17:21:57 -07:00
Anis Elleuch
14f0047295 fs: Remove fs meta lock when PutObject() fails (#4114)
Removing the fs meta lock file when PutObject() encounters any error
during its execution, such as upload getting permatuerly cancelled
by the client.
2017-04-14 12:06:24 -07:00
Krishna Srinivas
e6b2253da9 gateway: Fix help message for custom Azure Blob Storage endpoint. (#4113) 2017-04-14 11:02:43 -07:00
Krishnan Parthasarathi
ca64b86112 Return possible states a heal operation (#4045) 2017-04-14 10:28:35 -07:00
Karthic Rao
5f065e2a96 server: Fix CI build complaints (#4119)
- Ineffassign fixes.
- Spell check correction.
2017-04-14 08:00:04 -07:00
Harshavardhana
a7afa469e2 xl: Add stat calls to keep track of ignored errors. (#4117)
Such that in a situation where all errors were
ignored we need to reduce the errors using
readQuorum to get a consistent error value.

Without this change errors generated will
never be consistent with for an expected scenario.

For example in a 6 disk setup 1 disk is missing
and 5 do not have the volume (testbucket)

Without this change Stat() would result in different
errors depending on which disk died. Can cause
confusion to S3 client application.

This change addresses need to track type of
errors we ignored and bring readQuorum to
choose the maximally occuring as the value
of truth.
2017-04-14 01:46:16 -07:00
Bala FA
d103d5fb7c server: Error out if loopback addr is used for Distributed Erasure (#4105) 2017-04-12 20:27:24 -07:00
Harshavardhana
6683247080 tests: Fix the sopradic test failure in TestListObjectPartsDiskNotFound (#4107)
getBucketInfo() should keep track errors ignored,
such that in a situation where all errors were
ignored we need to reduce the errors using readQuorum
to get a consistent error value.

This is the problem we see with DiskNotFound test
disks are randomly removed.

Fixes #4095
2017-04-12 15:38:35 -07:00
Anis Elleuch
e4bd882f11 handlers: Ignore malformatted datetime type header (#4097)
Ignore headers, such as If-Modified-Since, If-Unmodified-Since, etc.. when they
are received with a format other than HTTP date.
2017-04-12 12:34:57 -07:00
Nitish Tiwari
4448285a83 Cleanup service docs (#4103)
- Remove windows service doc and add to Minio-service
- Remove Linux service doc as it is duplicated
2017-04-12 12:18:34 -07:00
Harshavardhana
952c618441 server: Fix a regression in printing startup banner. (#4100)
Octect based sorting was lost in the previous commit

de204a0a52

This PR fixes a regression - fixes #4099
2017-04-12 09:22:35 -07:00
Krishna Srinivas
c5249c35d3 gateway: Support for custom endpoint. (#4086) 2017-04-11 17:44:26 -07:00
Bala FA
de204a0a52 Add extensive endpoints validation (#4019) 2017-04-11 15:44:27 -07:00
Harshavardhana
1b1b9e4801 lock/rpc: change rpcPath to be called serviceEndpoint. (#4088)
This is a cleanup to ensure proper naming.
2017-04-11 10:25:21 -07:00
Rushan
1b0b2c1c76 Browser: Make login form browser auto-fill compatible (#4091) 2017-04-11 10:11:42 -07:00
koolhead17
7f5e037846 docs: Update docker quick-start guide and adds relevant project URLS (#4075) 2017-04-11 00:53:33 -07:00
Karthic Rao
929a13f33f Fix for writes from Apache Spark. (#4074)
- Due to usage of amazon SDK, spark expects md5sum of empty string to be
  returned when it does PUT on a directory.
- The fix returns md5sum of a empty string for the above mentioned case.
- This fixes the issue of Apache Spark not being able to write into Minio.
2017-04-10 19:51:23 -07:00
Krishna Srinivas
a4209c10ea signature-v4: Use sha256("") for calculating canonical request (#4064) 2017-04-10 09:58:08 -07:00
Harshavardhana
b927523223 server: Introduce a new env MINIO_REGION. (#4078)
This is implemented to be able to override region
through command line just like how access and
secret keys are provided.
2017-04-09 10:44:10 -07:00
Aditya Manthramurthy
604417baf4 Allow cluster to start when only n/2 servers are up (#4066)
Fixes #3234.

Relaxes the quorum requirement to start the object layer, and skips
quick-healing at start-up (as no write quorum is present).
2017-04-09 00:28:27 -07:00
Harshavardhana
6e9ac8db59 docker: Support docker swarm secrets. (#3977)
Fixes #3896
2017-04-08 01:43:40 -07:00
Harshavardhana
0497d5c342 api: SourceInfo should be populated in GET/HEAD notification. (#4073)
Refer https://github.com/minio/mc/issues/2073
2017-04-08 01:39:20 -07:00
Harshavardhana
6b4f368dfe notify: Webhook endpoints can fail, but we must start the server. (#4060)
Ignore any network errors when registering a webhook
notifier during Minio startup sequence. This way server
can be started even if the webhook endpoint is not available
and unreachable.
2017-04-08 01:13:55 -07:00
Nitish Tiwari
6507c30bbc Add steps to run Minio distributed on Windows (#4068) 2017-04-08 01:12:00 -07:00
Harshavardhana
f44f2e341c log: Dump signature request properly. (#4063)
Currently percent encoded strings are not properly encoded.

`s3%!!(MISSING)A(MISSING)`

Print properly as json encoded.

`s3%3AObjectCreated%3A%2A`
2017-04-07 14:37:32 -07:00
Krishnan Parthasarathi
60ea2b17ba Fix xml block syntax in admin-api Readme (#4062) 2017-04-07 03:38:01 -07:00
Harshavardhana
5ed1a8ad23 docs: macOS brew now refers to Minio fork (#4059) 2017-04-07 01:42:59 -07:00
Harshavardhana
27749c2124 admin/info: Add HTTPStats value as part of serverInfo() struct. (#4049)
Remove our counter implementation instead use atomic external
package which supports more types and methods.
2017-04-06 23:08:33 -07:00
Krishna Srinivas
1d99a560e3 refactor: extractSignedHeaders() handles headers removed by Go http server (#4054)
* refactor: extractSignedHeaders() handles headers removed by Go http server.
* Cleanup extractSignedHeaders() TestExtractSignedHeaders()
2017-04-05 17:00:24 -07:00
Krishna Srinivas
af82d27018 signature-v4: Support for transfer-encoding request header (#4053) 2017-04-05 15:08:33 -07:00
Anis Elleuch
f205689ff5 build: Fix compilation in 32 bits platforms (#4052)
go fails to build Minio under at least, armv6 and 386 due to some
inconsistencies in the type of one syscall variable in different
architectures. This PR casts that variable to uint64 to achieve
the desired consistency.
2017-04-05 11:17:59 -07:00
Harshavardhana
393c01d078 browser: Generate new UI assets. 2017-04-04 23:03:36 -07:00
Rajeesh C V
e9037b9d36 fix: add white space in storage usage section (#4038)
Fix for #3962 - add whitespace before storage usage details section in
 Browse component
2017-04-04 09:14:45 -07:00
Harshavardhana
4747adfcb4 fs: Enable returning ETag along with ListObjects() (#4042)
This is to comply with S3 behavior, we previously removed
reading `fs.json` for optimization reasons but we have a
reason to believe that providing ETag and using gjson
provides needed benefit of not having to deal with
unmarshalling overhead of golang stdlib.

Fixes #4028
2017-04-04 09:14:03 -07:00
Anis Elleuch
52d8f564bf sigv2: Unespace canonicalized resources values (#4034)
Values of canonicalized query resources should be unescaped before calculating
the signature. This bug is not noticed before because partNumber and uploadID
values in Minio doesn't have characters that need to be escaped.
2017-04-03 17:55:14 -07:00
Harshavardhana
3fe33e7b15 handler: simplify parsing valid location constraint. (#4040)
Separate out validating v/s parsing logic in
isValidLocationConstraint() into parseLocationConstraint()
and isValidLocation()

Additionally also set `X-Amz-Bucket-Region` as part of the
common headers for the clients to fallback on in-case of any
region related errors.
2017-04-03 14:50:09 -07:00
Krishnan Parthasarathi
4041e5f20d Provide mc-admin-heal command on start-up (#4031)
Healing of buckets, objects and incomplete uploads are implemented and
available via admin REST APIs. Additionally, it is available via mc admin 
sub-command. The warning is no longer relevant.

Fixes #4030
2017-04-03 14:24:25 -07:00
Krishnan Parthasarathi
96c46c15e7 madmin: Rename HealObjectResult to HealResult (#4035)
madmin.HealObjectResult is used in HealObject and HealUpload. It only
makes sense to rename it to HealResult.
2017-04-03 08:25:32 -07:00
Harshavardhana
3bf67668b6 sys/stats: return cgroup mem limit, fall back to sysinfo() (#4002)
This is necessary where in certain environments where
cgroup is used to limit memory usage of a container or
a particular process.

GetStats() is used by caching module to figure out the
optimal cacheable size in memory with cgroup limits
what sysinfo reports might not be the right value set
for a given process.

Fixes #4001
2017-04-02 10:46:16 -07:00
Anis Elleuch
e31e2c3bc2 doc: Explain how to create certificate chain file (#4032)
public.crt needs sometimes to have a chain certificate, this PR
explains how to construct public.crt when certificate are issued
by a certificate authority.
2017-04-02 04:47:56 -07:00
Harshavardhana
214279aa57 build: Reduce binary size by using -s -w (#4027)
Refer #3939
2017-04-01 01:06:16 -07:00
Harshavardhana
4de6b15fca vet: Fix all the go vet complaints (#4029)
```
go tool vet -atomic -bool -copylocks -nilfunc \
   -printf -shadow -rangeloops -unreachable \
   -unsafeptr -unusedresult cmd/
```
2017-04-01 01:06:06 -07:00
Krishnan Parthasarathi
2bd694dbc8 Add disksUnavailable healStatus const (#3990)
`disksUnavailable` healStatus constant indicates that a given object
needs healing but one or more of disks requiring heal are offline. This
can be used by admin heal API consumers to distinguish between a
successful heal and a no-op since the outdated disks were offline.
2017-03-31 17:55:15 -07:00
Aditya Manthramurthy
a2a8d54bb6 Add access format support for Elasticsearch notification target (#4006)
This change adds `access` format support for notifications to a
Elasticsearch server, and it refactors `namespace` format support.

In the case of `access` format, for each event in Minio, a JSON
document is inserted into Elasticsearch with its timestamp set to the
event's timestamp, and with the ID generated automatically by
elasticsearch. No events are modified or deleted in this mode.

In the case of `namespace` format, for each event in Minio, a JSON
document is keyed together by the bucket and object name is updated in
Elasticsearch. In the case of an object being created or over-written
in Minio, a new document or an existing document is inserted into the
Elasticsearch index. If an object is deleted in Minio, the
corresponding document is deleted from the Elasticsearch index.

Additionally, this change upgrades Elasticsearch support to the 5.x
series. This is a breaking change, and users of previous elasticsearch
versions should upgrade.

Also updates documentation on Elasticsearch notification target usage
and has a link to an elasticsearch upgrade guide.

This is the last patch that finally resolves #3928.
2017-03-31 14:11:27 -07:00
Harshavardhana
2040d32ef8 server/tls: Do not rely on a specific cipher suite (#4021)
Do not rely on a specific cipher suite instead let the
go choose the type of cipher needed, if the connection
is coming from clients which do not support forward
secrecy let the go tls handle this automatically based
on tls1.2 specifications.

Fixes #4017
2017-03-31 13:28:45 -07:00
Harshavardhana
f1015a5096 notifiers: Stop using url.Parse in validating address format. (#4011)
url.Parse() wrongly parses an address of format "address:port"
which is fixed in go1.8.  This inculcates a breaking change
on our end. We should fix this wrong usage everywhere so that
migrating to go1.8 eventually becomes smoother.
2017-03-31 04:47:40 -07:00
Aditya Manthramurthy
096427f973 Add deliveryMode parameter for AMQP notfication target (#4008)
Configuration migration was done.

Also adds documentation about AMQP configuration parameters.

Fixes #3982
2017-03-31 03:34:26 -07:00
Rushan
5cec6bd80d Browser: Use object name with prefix to delete sub-path objects (#4013) 2017-03-30 23:28:28 -07:00
Bala FA
6e9c91f43a fix: use its own lock in serverConfigV17 (#4014)
Previously serverConfigV17 used a global lock that made any instance of
serverConfigV17 depended on single global serverConfigMu.

This patch fixes by having individual lock per instances.
2017-03-30 22:26:24 -07:00
Bala FA
2df8160f6a server: handle command line and env variables at one place. (#3975) 2017-03-30 11:21:19 -07:00
Romain Bouyé
447fdd4097 docs: Fix typo in docs/config/README.md (#4009) 2017-03-30 11:09:15 -07:00
Harshavardhana
28c5a887de event: Set contentType as well under NotificationEvent. (#4003)
This is an enhancement change to to cater support all
the data fields present on the object. Currently
we only send a subset of data which object info
provides us.

It also helps us keep a full namespace mirror on
notification targets for efficient query.
2017-03-30 08:58:14 -07:00
Anis Elleuch
fbe8b3259d webhook: Add support of custom CAs (#4000) 2017-03-29 13:42:55 -07:00
Anis Elleuch
e2aba9196f obj-handlers: Rewrite src & dst path cmp in Copy() (#3998)
CopyObjectHandler() was incorrectly performing comparison
between destination and source object paths, which sometimes
leads to a lock race. This PR simplifies comparaison and add
one test case.
2017-03-29 09:21:38 -07:00
Aditya Manthramurthy
61b08137b0 Add access format support for Redis notification target (#3989)
This change adds `access` format support for notifications to a Redis
server, and it refactors `namespace` format support.

In the case of `access` format, a list is used to store Minio
operations in Redis. Each entry in the list is a JSON encoded list of
two items - the first is the Minio server timestamp of the event, and
the second is an object describing the operation that created/replaced
the object in the server.

In the case of `namespace` format, a hash is used. Entries in the hash
may be updated or removed if objects in Minio are updated or deleted
respectively. The field values in the Redis hash are JSON encoded.

Also updates documentation on Redis notification target usage.

Towards resolving #3928
2017-03-29 08:55:53 -07:00
Harshavardhana
1caad902cb config/path: Figure out absolute paths properly on windows. (#3996)
The following form of arguments such as

```
minio.exe -C some_dir server dir
```

has stopped working because of lack of handling of
absolute paths for config directory. Always calculate
absolute path for any relative paths on any operating
system.

The following fix converts all config directory relative
paths into absolute paths.

Fixes #3991
2017-03-29 08:55:33 -07:00
Nitish Tiwari
d99efa2c93 Update filename (#3995) 2017-03-28 23:14:28 -07:00
Krishna Srinivas
9ee83b89bb config: Appropriate error message when newer config file is found (#3972) 2017-03-28 18:41:16 -07:00
Nitish Tiwari
a8cb43926a Docker guide fix (#3992)
* Update Docker quick start guide

- Add Compose to orchestration section.

- Refer Swarm from Docker quick start guide.

- Add common Docker commands to quick start guide.

* Paragraph cleanup
2017-03-28 13:54:19 -07:00
Harshavardhana
b62cd8ed84 sign/streaming: Content-Encoding is not set in newer aws-java-sdks (#3986)
We can't use Content-Encoding to verify if `aws-chunked` is set
or not. Just use 'streaming' signature header instead.

While this is considered mandatory, on the contrary aws-sdk-java
doesn't set this value

http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html

```
Set the value to aws-chunked.
```

We will relax it and behave appropriately. Also this PR supports
saving custom encoding after trimming off the `aws-chunked`
parameter.

Fixes #3983
2017-03-27 17:02:04 -07:00
Anis Elleuch
1b3a517683 lock, Windows: O_CREAT infers only GENERIC_WRITE (#3981)
Avoid using GENERIC_ALL flag when creating a file since it could
return permission denied in some Windows versions.
2017-03-27 12:47:39 -07:00
Nitish Tiwari
a9c0f1e0a4 Added kernel tuning docs (#3921) 2017-03-27 11:29:04 -07:00
Aditya Manthramurthy
a099319e66 Support access format for database notification targets (#3953)
* Add configuration parameter "format" for db targets and perform
  configuration migration.
* Add PostgreSQL `access` format: This causes Minio to append all events
  to the configured table. Prefix, suffix and event filters continue
  to be supported for this mode too.
* Update documentation for PostgreSQL notification target.
* Add MySQL `access` format: It is very similar to the same format for
  PostgreSQL.
* Update MySQL notification documentation.
2017-03-27 11:27:25 -07:00
Bala FA
6e63904048 browser-flag: wrapped bool type denotes browser on/off flag. (#3963)
Statically typed BrowserFlag prevents any arbitrary string value
usage. The wrapped bool marshals/unmarshals JSON according to the
typed value ie string value "on" represents boolean true and "off" as
boolean false.
2017-03-26 12:00:27 -07:00
Harshavardhana
565ac4c861 tests: use url.QueryEscape() when dealing with url query params. (#3974)
This is to keep the portability and also avoid errors that
might occur using the functions written for URL resource name
Since query param values have different escaping requirements.
2017-03-26 11:56:17 -07:00
Harshavardhana
a4ecd8bca2 docs: Add config directory documentation/guide. (#3889) 2017-03-25 02:34:04 -07:00
Harshavardhana
28eff0f6c1 build: Improve build messaging, say where we built Minio. (#3973) 2017-03-25 00:33:57 -07:00
Krishnan Parthasarathi
c27ece409b heal: Check if all parts are available and valid (#3967)
In the algorithm to check if an object requires healing, in addition to
checking if all disks have xl.json present we should check if all parts
of the object are present and have valid blake2b checksums.

Also fixed a minor compilation error in heal-objects-list.go.
2017-03-24 08:40:44 -07:00
Dee Koder
8c0ce2fee9 docs: Removed space from code blocks. (#3965) 2017-03-23 19:06:31 -07:00
Bala FA
d3cb79a57c Refactor logger (#3924)
This patch fixes below

* Previously fatalIf() never writes log other than first logging target.
* quiet flag is not honored to show progress messages other than startup messages.
* Removes console package usage for progress messages.
2017-03-23 16:36:00 -07:00
Anis Elleuch
11e15f9b4c config: Remove level in console/file loggers (#3938)
Also rename fileName field in file logger to filename
2017-03-23 08:27:22 -07:00
Krishnan Parthasarathi
4e92b2ecb8 Fix listDirHealFactory merging of entries across disks (#3959)
For listing of objects needing heal, we list all objects present on all
the disks and return the set union. We were incorrectly dropping objects
that weren't already seen in disks so far.

Sample directory layout of disks in a 4-disk setup:
`/tmp/1`, `/tmp/2`, `/tmp/3`, `/tmp/4` are directories used as disks here.
`test` is the bucket, `obj1` and obj2` are the objects.
```
/tmp/1/test
└── obj2
    ├── part.1
    ├── part.2
    └── xl.json
/tmp/2/test
└── obj1
    ├── part.1
    ├── part.2
    └── xl.json
/tmp/3/test
├── obj1
│   ├── part.1
│   ├── part.2
│   └── xl.json
└── obj2
    ├── part.1
    ├── part.2
    └── xl.json
/tmp/4/test
[This is empty]

```
2017-03-23 08:24:59 -07:00
koolhead17
80b83a51a3 Docs: Fix for Self signed certificate. (#3957) 2017-03-23 08:20:39 -07:00
Krishna Srinivas
777d12d928 browser: update ui-assets.go (#3956) 2017-03-22 23:17:14 -07:00
Krishnan Parthasarathi
607c8a9611 Add sourceInfo to NotificationEvent (#3937)
This change adds information like host, port and user-agent of the
client whose request triggered an event notification.

E.g, if someone uploads an object to a bucket using mc. If notifications
were configured on that bucket, the host, port and user-agent of mc
would be sent as part of event notification data.

Sample output:
```
"source": {
          "host": "127.0.0.1",
          "port": "55808",
          "userAgent": "Minio (linux; amd64) minio-go/2.0.4 mc ..."
}
```
2017-03-22 18:44:35 -07:00
Anis Elleuch
7f5a5b5e9d config: Do not validate creds when set via env (#3955)
It is useless to validate access/secret keys stored in
config file when the user sets them in the environment.
2017-03-22 16:11:58 -07:00
Bala FA
d4ca2ee1a3 pkg/quick: add Save() function and other enhancements. (#3951)
* Add a new function Save() which saves given configuration into given file.
* Simplify Load() function.
* Remove unused CheckVersion().
* CheckData() is a private function now.
* quick_test.go is part of quick package now.
* minio server uses top level quick.Load() and quick.Save() functions.
2017-03-22 10:23:25 -07:00
Krishnan Parthasarathi
417ec0df56 HealObject should succeed when only N/2 disks have data (#3952) 2017-03-22 10:15:16 -07:00
Rushan
fbfb4fc5a0 Browser: Use polyfill to support Object.assign in IE11 (#3942) 2017-03-21 13:48:07 -07:00
koolhead17
97dc34fe06 docs: Fix for README.md markdown table. (#3948) 2017-03-21 12:02:39 -07:00
Krishnan Parthasarathi
9f9ba1e984 XL: Return the right error (#3944) 2017-03-21 10:33:25 -07:00
Krishnan Parthasarathi
13c4ce3617 Add notification for object access via GET/HEAD (#3941)
The following notification event types are available for all targets,
      s3:ObjectAccessed:Get
      s3:ObjectAccessed:Head
      s3:ObjectAccessed:*
2017-03-21 10:32:17 -07:00
Krishnan Parthasarathi
181e002c56 pkg/madmin: Set UploadID in ListUploadsHeal (#3945)
Without this fix, `mc admin heal -I` wouldn't be able to heal ongoing
uploads. `mc` depends on `ListUploadsHeal` API to identify ongoing
uploads to heal given a bucket and an object.
2017-03-21 10:32:02 -07:00
Anis Elleuch
9d6e226692 heal: Set truncate when no more walk entries (#3932) 2017-03-20 15:31:25 -07:00
Krishnan Parthasarathi
eb02261642 XL: Don't return ignored errors in listDirFactory (#3935)
Previously, erasure backend's `listDirFactory` may return errors which
were explicitly ignored. With this change, it returns nil. Superfluous
checks at higher-layers for ignored errors are removed as well.
2017-03-20 11:09:05 -07:00
Pawan Rawal
1396e91dd1 Update link for downloading minio server. (#3934)
This and the link for downloading Minio server at other places in the docs seems to be broken. I suppose this happened while updating the name of the page (which updated the url) in Doctor docs. 
Might be nice for Doctor to update internal links if the name of a page is changed in a background job.
2017-03-19 18:26:23 -07:00
Bala FA
7ebf11b202 words: new package Damerau Levenshtein distance function. (#3929) 2017-03-19 14:23:05 -07:00
Bala FA
1c97dcb10a Add UTCNow() function. (#3931)
This patch adds UTCNow() function which returns current UTC time.

This is equivalent of UTCNow() == time.Now().UTC()
2017-03-18 11:28:41 -07:00
Anis Elleuch
3a6111eff5 admin: Export HealStatus data type (#3930)
`healStatus` can be returned to the API caller. This commit will help
developers to declare a variable with HealStatus type.
2017-03-18 11:27:27 -07:00
Aditya Manthramurthy
2463ae243a Add support for MySQL notifications (fixes #3818) (#3907)
As a new configuration parameter is added, configuration version is
bumped up from 14 to 15.

The MySQL target's behaviour is identical to the PostgreSQL: rows are
deleted from the MySQL table on delete-object events, and are
created/updated on create/over-write events.
2017-03-17 09:29:17 -07:00
Krishnan Parthasarathi
c192e5c9b2 Implement heal-upload admin API (#3914)
This API is meant for administrative tools like mc-admin to heal an
ongoing multipart upload on a Minio server.  N B This set of admin
APIs apply only for Minio servers.

`github.com/minio/minio/pkg/madmin` provides a go SDK for this (and
other admin) operations.  Specifically,

  func HealUpload(bucket, object, uploadID string, dryRun bool) error

Sample admin API request:
POST
/?heal&bucket=mybucket&object=myobject&upload-id=myuploadID&dry-run
- Header(s): ["x-minio-operation"] = "upload"

Notes:
- bucket, object and upload-id are mandatory query parameters
- if dry-run is set, API returns success if all parameters passed are
  valid.
2017-03-17 09:25:49 -07:00
Nitish Tiwari
d4eea224d4 Remove white spaces (#3922) 2017-03-17 09:23:22 -07:00
Nitish Tiwari
e55421ebdd Fixed Docker compose link (#3920) 2017-03-17 01:06:11 -07:00
Dee Koder
0a5d57a91e docs: Update gateway doc with roadmap section. (#3918) 2017-03-16 18:25:01 -07:00
Minio Trusted
6e7d33df20 Bump to new release 2017-03-16 14:52:42 -07:00
Minio Trusted
5311eb22fd Fix dockerfile 2017-03-16 14:49:30 -07:00
Krishna Srinivas
cea4cfa3a8 Implement S3 Gateway to third party cloud storage providers. (#3756)
Currently supported backend is Azure Blob Storage.

```
export MINIO_ACCESS_KEY=azureaccountname
export MINIO_SECRET_KEY=azureaccountkey
minio gateway azure
```
2017-03-16 12:21:58 -07:00
Anis Elleuch
8426cf9aec config: Accept more address format + unit test (#3915)
checkURL() is a generic function to check if a passed address
is valid. This commit adds support for addresses like `m1`
and `172.16.3.1` which is needed in MySQL and NATS. This commit
also adds tests.
2017-03-16 11:44:01 -07:00
Harshavardhana
f3334159a4 config: Relax browser and region to be empty. (#3912)
- If browser field is missing or empty
  then default to "on".

- If region field is empty or missing then
  default to "us-east-1" (S3 spec behavior)
2017-03-16 11:06:17 -07:00
Harshavardhana
3edff1501e Vendor upstream redis library instead of our fork. (#3913)
We forked the upstream to address a build issue on
go 1.6 - that is long done and we don't need
to manage our forks anymore.
2017-03-16 08:22:47 -07:00
Bala FA
21d73a3eef Simplify credential usage. (#3893) 2017-03-16 00:16:06 -07:00
Krishnan Parthasarathi
051f9bb5c6 Implement list uploads heal admin API (#3885) 2017-03-16 00:15:06 -07:00
Harshavardhana
6509589adb Use canonicalETag helper wherever needed. (#3910) 2017-03-15 20:48:49 -07:00
Anis Elleuch
ae4361cc45 config: Check for duplicated entries in all scopes (#3872)
Validate Minio config by checking if there is double json key
in any scope level. The returned error contains the json path
to the duplicated key.
2017-03-15 16:30:34 -07:00
Krishna Srinivas
cad0d0eb7a browser: Update ui-assets.go (#3902)
fixes #3898
2017-03-15 11:32:51 -07:00
Nitish Tiwari
ba0c11757e Added that no special config changes reqd (#3906)
Added a line saying no special config changes are required for Shared mode. Also the previous `Why Shared Backend` and current `Use cases` are already merged. This fixes the comment: https://github.com/minio/minio/pull/3888#discussion_r105573494
2017-03-15 08:22:03 -07:00
Krishna Srinivas
96050c1e21 browser: Do not show "Loading..." if there are no buckets. (#3904) 2017-03-14 19:07:23 -07:00
Rushan
a27d1b3d86 Browser: Use babel-polyfill to support new ES6 built-ins in older browsers (#3900) 2017-03-14 15:02:20 -07:00
Anis Elleuch
a5e60706a2 xl,fs: Return 404 if object ends with a separator (#3897)
HEAD Object for FS and XL was returning invalid object name when
an object name has a trailing slash separator, this PR changes the
behavior and will always return 404 object not found, this guarantees
a better compatibility with S3 spec.
2017-03-13 22:20:46 -07:00
Harshavardhana
5f7565762e api: postPolicy cleanup. Simplify the code and re-use. (#3890)
This change is cleanup of the postPolicyHandler code
primarily to address the flow and also converting
certain critical parts into self contained functions.
2017-03-13 14:41:13 -07:00
Nitish Tiwari
3314501f19 Simplify shared mode document (#3888) 2017-03-12 16:17:03 -07:00
Harshavardhana
9d53a646a1 Simplify the title for orchestration and some words. (#3887) 2017-03-11 02:17:03 -08:00
Nitish Tiwari
5e0032e165 Update links (#3886) 2017-03-11 00:11:07 -08:00
Harshavardhana
305952d734 browser: Update ui-assets with new changes. 2017-03-10 15:13:26 -08:00
Rushan
e77885d671 Browser: Add Object.assign polyfill to support older browsers (#3884) 2017-03-10 14:30:23 -08:00
Nitish Tiwari
2410eb281e Update kafkacat command with consumer flag (#3882) 2017-03-10 09:01:59 -08:00
Harshavardhana
e54025805f browser: update ui-assets with new changes. 2017-03-09 15:26:25 -08:00
Anis Elleuch
d602495600 madmin: Do not require SSL to set credentials (#3879)
We need to relax this requirement and let the client decides
if it can allow to set credentials API over plain connection.
2017-03-09 14:08:33 -08:00
Nitish Tiwari
03937e7554 Added documentation for orchestration platforms (#3684) 2017-03-09 14:06:51 -08:00
Harshavardhana
e3b627a192 docker: Add ARM64 image build support (#3876) 2017-03-09 13:57:27 -08:00
Rushan
ccc3349f0c Browser: Show complete bucket name by removing the ellipsis cut off (#3877) 2017-03-09 10:42:38 -08:00
Harshavardhana
85cbd875fc cleanup: All conditionals simplified under pkg. (#3875)
Address all the changes reported/recommended by
`gosimple` tool.
2017-03-09 10:13:30 -08:00
Harshavardhana
43317530d5 Fix odd shadowing bug in XL init. (#3874)
Fixes #3873
2017-03-08 20:42:45 -08:00
Bala FA
8a9852220d Make unit testable cert parsing functions. (#3863) 2017-03-08 19:20:01 -08:00
Harshavardhana
47ac410ab0 Code cleanup - simplify server side code. (#3870)
Fix all the issues reported by `gosimple` tool.
2017-03-08 10:00:47 -08:00
Harshavardhana
433225ab0d docker: ca-certificates should not be removed. (#3868) 2017-03-07 16:30:19 -08:00
Harshavardhana
3e655a2c85 docker: Add ARM docker container dockerfile. (#3574) 2017-03-07 15:15:05 -08:00
Anis Elleuch
a2eae54d11 xl: Respect min. space by checking PrepareFile err (#3867)
It was possible to upload a big file which overcomes the minimal
disk space limit in XL, PrepareFile was actually checking for disk
space but we weren't checking its returned error. This patch fixes
this behavior.
2017-03-07 14:48:56 -08:00
Anis Elleuch
79e0b9e69a Relax minio server start when disk threshold is reached and adds space check in FS (#3865)
* fs: Rename tempObjPath variable in fsCreateFile()
* fs/posix: Factor checkDiskFree() function
* fs: Add disk free check in fsCreateFile()
* posix: Move free disk check to createFile()
* xl: Relax free disk check in POSIX initialization
* fs: checkDiskFree checks for space to store data
2017-03-07 12:25:40 -08:00
Krishna Srinivas
29ff9674a0 browser: Humanize expiry time for Share-Object. (#3861) 2017-03-06 20:13:52 -08:00
Bala FA
bff4d29415 Remove commands and commandsTree global variables. (#3855) 2017-03-06 19:35:26 -08:00
Krishna Srinivas
436db49bf3 Share object expiry value modification modal was not working (#3860) 2017-03-06 18:50:25 -08:00
Rushan
966818955e Browser: Implement multiple object delete (#3859) 2017-03-06 15:43:43 -08:00
Harshavardhana
e49efcb9d9 xl: quickHeal heal bucket only when needed. (#3854)
This improves the startup time significantly
for clusters which have lot of buckets.

Also fixes a bug where `.minio.sys` is created
on disks which do not have `format.json`
2017-03-06 02:00:15 -08:00
Harshavardhana
6f931d29c4 rpm: Add RPM spec for minio build. (#3853)
Currently the package is built and hosted at

https://copr.fedorainfracloud.org/coprs/minio/minio/

To enable minio repo one has to download.

Fedora - 25
https://copr.fedorainfracloud.org/coprs/minio/minio/repo/fedora-25/minio-minio-fedora-25.repo

Fedora - 26
https://copr.fedorainfracloud.org/coprs/minio/minio/repo/fedora-26/minio-minio-fedora-26.repo

Enables for both i386 and x86_64.

Fixes #3576
2017-03-05 13:09:31 -08:00
Krishnan Parthasarathi
e3fd4c0dd6 XL: Make listOnlineDisks and outDatedDisks consistent w/ each other. (#3808) 2017-03-04 14:53:28 -08:00
Harshavardhana
b05c1c11d4 browser: Update UI assets with new changes. 2017-03-03 18:05:02 -08:00
Krishna Srinivas
0bae3330e8 browser: Send correct arguments for RemoveObjects web handler. (#3848)
Fixes #3839
2017-03-03 17:07:17 -08:00
Harshavardhana
05e53f1b34 api: CopyObjectPart was copying wrong offsets due to shadowing. (#3838)
startOffset was re-assigned to '0' so it would end up
copying wrong content ignoring the requested startOffset.

This also fixes the corruption issue we observed while
using docker registry.

Fixes https://github.com/docker/distribution/issues/2205

Also fixes #3842 - incorrect routing.
2017-03-03 16:32:04 -08:00
Anis Elleuch
0c8c463a63 tests: Fix web handlers testing with faulty disks (#3845) 2017-03-03 15:46:28 -08:00
Aditya Manthramurthy
6df7bc42b8 Fix check for bucket name: (#3832)
* Do not allow bucket names with adjacent hypen and periods.
* Improve performance by eliminating the usage of regular expressions.
2017-03-03 10:23:41 -08:00
Anis Elleuch
6c00a57a7c quick: Add yaml format support (#3833)
quick Save() and Load() infers config file's format from
file name extension.
2017-03-03 10:22:09 -08:00
Harshavardhana
bc52d911ef api: Increase the maximum object size limit from 5GiB to 16GiB. (#3834)
The globalMaxObjectSize limit is instilled in S3 spec perhaps
due to certain limitations on S3 infrastructure. For minio we
don't have such limitations and we can stream a larger file
instead.

So we are going to bump this limit to 16GiB.

Fixes #3825
2017-03-03 10:14:17 -08:00
Anis Elleuch
28c53a3555 obj: Make checkBucketExist() returns all errors (#3843)
This function was returning BucketNotFound for all errors
which at least hides the fact that disks could be corrupted.
This commit fixes the behavior by returning all errors that,
are, by the way, Object API errors.
2017-03-03 10:12:43 -08:00
Harshavardhana
e5d4e7aa9d web: Validate if bucket names are reserved (#3841)
Both '.minio.sys' and 'minio' should be never allowed
to be created from web-ui and then fail to list it
by filtering them out.

Fixes #3840
2017-03-03 03:01:42 -08:00
Anis Elleuch
cddc684559 admin: Set Config returns errSet and errMsg (#3822)
There is no way to see if a node encountered an error
when trying to set a new config set, this commit adds
a bool errSet field.
2017-03-03 02:53:48 -08:00
Zejun Li
32d0d3d4ac Enhanced newObjectLayerFn (#3837) 2017-03-03 01:07:45 -08:00
Bala FA
98d17d2a97 Remove globalQuiet and globalConfigDir global variables (#3830) 2017-03-02 14:21:30 -08:00
Bala FA
208dd15245 Remove globalMaxCacheSize and globalCacheExpiry variables (#3826)
This patch fixes below

* Remove global variables globalMaxCacheSize and globalCacheExpiry.
* Make global variables into constant in objcache package.
2017-03-02 10:34:37 -08:00
Anis Elleuch
a179fc9658 quick: Simplify Load() and CheckVersion() (#3831) 2017-03-02 10:29:06 -08:00
Zejun Li
d1afd16955 Using RWMutex to guard closing and listeners (#3829) 2017-03-02 10:00:22 -08:00
Bala FA
2348ae7a19 Make default values as constants (#3828) 2017-03-02 04:58:39 -08:00
Bala FA
480ea826dc Move rlimit functions into sys package. (#3824)
This patch addresses below

* go build works for bsd family
* probe total RAM size for bsd family
* make unit testable functions
2017-03-01 21:51:57 -08:00
Aditya Manthramurthy
09e9fd745c Close client connection after checking for release update (#3820) 2017-03-01 09:18:55 -08:00
Anis Elleuch
77c1998a38 config: Fix creating new config with wrong version (#3821)
Simplify a little config code to avoid making mistake
next time.
2017-03-01 09:17:04 -08:00
Krishna Srinivas
91cf54f895 web-handlers: Support removal of multiple objects at once. (#3810) 2017-02-28 19:07:28 -08:00
Karthic Rao
2b0ed21f08 tests: Fix test server init - cleanup (#3806) 2017-02-28 18:05:52 -08:00
Harshavardhana
472fa4a6ca api: Multi object delete should be protected. (#3814)
Add missing protection from deleting multiple objects
in parallel. Currently we are deleting objects without
proper locking through this API.

This can cause significant amount of races.
2017-02-28 18:00:24 -08:00
Bala FA
097cec676a fix: Set globalMaxCacheSize to allowable value. (#3816)
If memory resource limit and total RAM are more than 8GiB, either 50%
of memory resource limit or total RAM is set to globalMaxCacheSize.
2017-02-28 16:51:52 -08:00
Rushan
fcad4a44fd Browser: Remove duplicate object entries while sorting (#3813) 2017-02-28 13:11:34 -08:00
Anis Elleuch
9b3c014bab config: Add browser parameter (#3807)
browser new parameter receives "on" or "off" parameter which is similar
to MINIO_BROWSER
2017-02-27 14:59:53 -08:00
Krishnan Parthasarathi
c9619673fb Implement SetConfig admin API handler. (#3792) 2017-02-27 11:40:27 -08:00
Anis Elleuch
dce0345f8f Set disk to nil after write which needs quorum (#3795)
Ignore a disk which wasn't able to successfully perform an action to
avoid eventual perturbations when the disk comes back in the middle
of write change.
2017-02-26 11:58:32 -08:00
Anis Elleuch
461b2bbd37 admin: Move SetCredentials from Service to Generic (#3805)
Setting credentials doesn't belong to service management API
anymore.
2017-02-25 11:06:08 -08:00
Bala FA
69777b654e event: use common initialization logic (#3798)
Previously creating and adding targets for each notification type was
repeated.  This patch fixes it.
2017-02-24 18:27:52 -08:00
Harshavardhana
70d2cb5f4d rpc: Remove time check for each RPC calls. (#3804)
This removal comes to avoid some redundant requirements
which are adding more problems on a production setup.

Here are the list of checks for time as they happen

 - Fresh connect (during server startup) - CORRECT
 - A reconnect after network disconnect - CORRECT
 - For each RPC call - INCORRECT.

Verifying time for each RPC aggravates a situation
where a RPC call is rejected in a sequence of events
due to enough load on a production setup. 3 second
might not be enough time window for the call to be
initiated and received by the server.
2017-02-24 18:26:56 -08:00
Harshavardhana
cff45db1b9 cli: Use ADDRESS:PORT to clarify --address behavior (#3803)
Currently we document as IP:PORT which doesn't provide
if someone can use HOSTNAME:PORT. This is a change
to clarify this by calling it as ADDRESS:PORT which
encompasses both a HOSTNAME and an IP.

Fixes #3799
2017-02-24 14:19:20 -08:00
Harshavardhana
bcc5b6e1ef xl: Rename getOrderedDisks as shuffleDisks appropriately. (#3796)
This PR is for readability cleanup

- getOrderedDisks as shuffleDisks
- getOrderedPartsMetadata as shufflePartsMetadata

Distribution is now a second argument instead being the
primary input argument for brevity.

Also change the usage of type casted int64(0), instead
rely on direct type reference as `var variable int64` everywhere.
2017-02-24 09:20:40 -08:00
Harshavardhana
25b5a0534f browser: Update ui-assets and fix the copyright header. (#3790) 2017-02-22 17:27:26 -08:00
Rushan
52d6678bf0 Browser: Implement multi select user interface for object listings (#3730) 2017-02-22 14:51:38 -08:00
Nitish Tiwari
d8950ba7c5 Added server times note and fix Notes rendering for Doctor. (#3787) 2017-02-22 02:12:03 -08:00
Harshavardhana
cc28765025 xl/multipart: Make sure to delete temp renamed object. (#3785)
Existing objects before overwrites are renamed to
temp location in completeMultipart. We make sure
that we delete it even if subsequenty calls fail.

Additionally move verifying of parent dir is a
file earlier to fail the entire operation.

Ref #3784
2017-02-21 19:43:44 -08:00
Harshavardhana
fe86319c56 ci: For windows builds stick to go1.7.5 (#3786) 2017-02-21 17:24:11 -08:00
Harshavardhana
99a12613a3 update: For source builds look for absolute path. (#3780)
os.Args[0] doesn't point to absolute path we need
use exec.LookPath to find the absolute path before
sending os.Stat().
2017-02-21 01:32:05 -08:00
Nitish Tiwari
097dd7418a Remove unused erasure diagram (#3783) 2017-02-20 21:01:08 -08:00
Nitish Tiwari
a7d3ea8c15 Update erasure code image (#3782) 2017-02-20 20:12:21 -08:00
Krishnan Parthasarathi
2745bf2f1f Implement ServerConfig admin REST API (#3741)
Returns a valid config.json of the setup. In case of distributed
setup, it checks if quorum or more number of nodes have the same
config.json.
2017-02-20 12:58:50 -08:00
Anis Elleuch
70d825c608 doc: Small rewrite of bucket events notif intro (#3775) 2017-02-20 12:07:27 -08:00
Harshavardhana
6b68c0170f For streaming signature do not save content-encoding in PutObject() (#3776)
Content-Encoding is set to "aws-chunked" which is an S3 specific
API value which is no meaning for an object. This is how S3
behaves as well for a streaming signature uploaded object.
2017-02-20 12:07:03 -08:00
Aditya Manthramurthy
0a905e1a8a Fix rabbitmq reconnect problem (#3778) 2017-02-20 12:05:21 -08:00
Harshavardhana
9eb8e375c5 cli: Make sure to add --help flag for subcommands. (#3773)
--help is now back and prints properly with command
help template.
2017-02-19 20:46:06 -08:00
Harshavardhana
7ea1de8245 copyObject: Be case sensitive for windows only server. (#3766)
For case sensitive platforms we should honor case.

Fixes #3765

```
1) python s3cmd -c s3cfg_localminio put logo.png s3://testbucket/xyz/etc2/logo.PNG

2) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58     22059   s3://testbucket/xyz/etc2/logo.PNG

3) python s3cmd -c s3cfg_localminio cp s3://testbucket/xyz/etc2/logo.PNG s3://testbucket/xyz/etc2/logo.png
remote copy: 's3://testbucket/xyz/etc2/logo.PNG' -> 's3://testbucket/xyz/etc2/logo.png'

4) python s3cmd -c s3cfg_localminio ls s3://testbucket/xyz/etc2/
2017-02-18 10:58     22059   s3://testbucket/xyz/etc2/logo.PNG
2017-02-18 11:10     22059   s3://testbucket/xyz/etc2/logo.png
```
2017-02-18 13:41:59 -08:00
Anis Elleuch
54a18592e9 flags: Fix --version output (#3772) 2017-02-18 13:41:33 -08:00
Anis Elleuch
7e84c7427d server-mux: Rewrite graceful shutdown mechanism (#3771)
Old code uses waitgroup Add() and Wait() in different threads,
which eventually can lead to a race.
2017-02-18 13:28:54 -08:00
Bala FA
d12f3e06b1 config-old: Use interface to avoid code repetition. (#3769) 2017-02-18 10:45:37 -08:00
Harshavardhana
0137ff498a auth/rpc: Token can be concurrently edited protect it. (#3764)
Make sure we protect when we access `authToken` in authClient.

Fixes #3761
2017-02-18 03:15:42 -08:00
Harshavardhana
34d9a6b46a Make sure client initializes to proper lock RPC path. (#3763)
Fixes a regression introduced in previous commit.
2017-02-18 02:52:11 -08:00
Harshavardhana
50b4e54a75 fs: Do not return reservedBucket names in ListBuckets() (#3754)
Make sure to skip reserved bucket names in `ListBuckets()`
current code didn't skip this properly and also generalize
this behavior for both XL and FS.
2017-02-16 14:52:14 -08:00
Harshavardhana
8816b08aae Fix the systemd config path to the new URL 2017-02-15 21:28:06 -08:00
Harshavardhana
271e3ecde5 Fix tests from cli changes 2017-02-15 18:05:55 -08:00
Minio Trusted
83abb310b4 Bump to new release 2017-02-15 17:51:19 -08:00
Minio Trusted
3d98311d9f Merge branch 'master' into release 2017-02-15 17:47:12 -08:00
Harshavardhana
611bd68739 cli: Add new features from CLI.
use `.HelpName` for template to pass down
proper command names. Also treat flags
to be optional in all of them.
2017-02-15 17:45:11 -08:00
Harshavardhana
25eeb88a8f cli: Bring upstream changes to minio server. 2017-02-15 17:30:31 -08:00
Minio Trusted
d44e38b462 Bump to new release 2017-02-15 15:09:15 -08:00
Minio Trusted
8948c972ee Merge branch 'master' into release 2017-02-15 15:07:51 -08:00
Krishna Srinivas
3e770defae browser: Update ui assets with new changes (#3751) 2017-02-15 14:44:34 -08:00
Krishna Srinivas
ae4656e699 browser: Remove currentPath prefix in object names from list result. (#3750) 2017-02-15 13:07:47 -08:00
Harshavardhana
f6a9e690bf browser: Update ui assets with new changes. 2017-02-15 11:29:23 -08:00
Anis Elleuch
7f86a21317 admin: Add ServerInfo API() (#3743) 2017-02-15 10:45:45 -08:00
Harshavardhana
fb39c7c26b sRPC/client: Properly trim storageRPCPath for actual disk path. (#3749)
Never print internal RPC endpoint paths.
2017-02-15 03:47:47 -08:00
Harshavardhana
13c3b9cbcb main/cli: Don't print hidden flags/commands in help template. (#3748)
Always use .VisibleFlags and .VisibleCommands to not print
Hidden flags as expected from help template.
2017-02-15 02:25:38 -08:00
Bala FA
602dac8773 mainUpdate: refactor to handle quiet flag properly (#3744) 2017-02-15 00:31:00 -08:00
Dee Koder
c6e76160ad Update README.md 2017-02-14 14:20:29 -08:00
Bala FA
a53b909fcd mainVersion: Remove minio init and quiet handling (#3739) 2017-02-13 00:24:34 -08:00
Harshavardhana
22909c849e objcache: Return io.ReaderAt to avoid Seeking and Reading. (#3735) 2017-02-11 17:17:58 -08:00
Bala FA
440866d26c Move go version check into main() (#3734)
Previously the check was done in init().  This patch moves into main
for unit testable friendly function.
2017-02-11 01:27:27 -08:00
Krishna Srinivas
25b936c369 browser: Implement infinite scrolling for object listing. (#3720)
fixes #2831
2017-02-10 22:54:42 -08:00
Nitish Tiwari
8f66cfa316 Fix https://github.com/minio/minio/issues/3453 (#3733) 2017-02-10 22:53:17 -08:00
Bala FA
8912b6bf3b trie: new package (#3729)
This implements a simple trie tree for minio server/tools.
This package borrows idea from
https://godoc.org/golang.org/x/text/internal/triegen.
2017-02-10 11:51:41 -08:00
Karthic Rao
a799351926 Fix macOS builds (#3728) 2017-02-10 00:37:22 -08:00
Harshavardhana
9df01035da Remove XL references in public docs to Erasure. (#3725)
Ref #3722
2017-02-09 23:26:44 -08:00
Krishnan Parthasarathi
1ad96ee09f Handle uptime for single-node instances differnt from distributed setup (#3726) 2017-02-09 20:38:14 -08:00
Harshavardhana
1b4bb94ac4 config: setter/getter for Notifier and Logger into its own struct. (#3721)
This is an attempt cleanup code and keep the top level config
functions simpler and easy to understand where as move the
notifier related code and logger setter/getter methods as part
of their own struct.

Locks are now held properly not globally by configMutex, but
instead as private variables.

Final fix for #3700
2017-02-09 15:20:54 -08:00
Andreas Auernhammer
f38222c0cc update the blake2b implementation (#3724)
Fixes a performance bug caused by SSE-AVX register savings on amd64.
2017-02-09 15:01:00 -08:00
Anis Elleuch
c9b1468c3b Presigend Post: Error out when File is not found (#3723)
Follow S3 behavior when no File is sent in the presigned
post request form.
2017-02-09 12:37:32 -08:00
Krishna Srinivas
6800902b43 web-handlers: Implement API to download files as a zip file. (#3715) 2017-02-08 23:39:08 -08:00
Krishnan Parthasarathi
e5773e11c6 Make minio server compile on OpenBSD, NetBSD, Solaris (#3719) 2017-02-08 22:27:35 -08:00
Nitish Tiwari
0c7694894b Added spaces for formatting (#3717) 2017-02-08 00:51:08 -08:00
Krishnan Parthasarathi
ce9aa2f2b2 Add uptime to ServiceStatus (#3690) 2017-02-08 00:13:02 -08:00
Nitish Tiwari
7547f3c8a3 Added Webhook notification details (#3706) 2017-02-07 23:19:32 -08:00
Harshavardhana
31dff87903 Honor envs properly for access and secret key. (#3703)
Also changes the behavior of `secretKeyHash` which is
not necessary to be sent over the network, each node
has its own secretKeyHash to validate.

Fixes #3696
Partial(fix) #3700 (More changes needed with some code cleanup)
2017-02-07 12:51:43 -08:00
Anis Elleuch
fd72c21e0e tests: Reduce two functions work to gain test time (#3712)
TestListObjectsHeal and TestFSShutdown takes around 3 min,
this PR reduces the number of created test objects
2017-02-07 12:51:23 -08:00
Harshavardhana
310bf5bd36 auth/rpc: Make auth rpc client retry configurable. (#3695)
Currently the auth rpc client defaults to to a maximum
cap of 30seconds timeout. Make this to be configurable
by the caller of authRPCClient during initialization, if no
such config is provided then default to 30 seconds.
2017-02-07 02:16:29 -08:00
Harshavardhana
a170e44689 Let hijacker and flusher interfaces to be reflected. (#3709)
Ideally here if the interface is not found it would
fail the server, as it should be because without these
we can't even have a working server in the first place.

Just like how it fails in master invariably inside Go
net/http code path.

Fixes #3708
2017-02-06 23:52:47 -08:00
Anis Elleuch
70e70446bb signv4: Read always returns EOF when stream ends (#3692)
When EOF is reached, further calls of Read() doesn't return io.EOF
but continue to work as it expects to have more data, this PR fixes
the behavior
2017-02-06 14:19:27 -08:00
Krishna Srinivas
45d9cfa0c5 signature-v4: stringToSign and signingKey should use Scope's date. (#3688)
fixes #3676
2017-02-06 13:09:09 -08:00
Anis Elleuch
93fd269329 stats: Add network and http statisics (#3686)
Network: total bytes of incoming and outgoing server's data
by taking advantage of our ConnMux Read/Write wrapping

HTTP: total number of different http verbs passed in http
requests and different status codes passed in http responses.
This is counted in a new http handler.
2017-02-06 09:29:53 -08:00
Harshavardhana
6717a0b68c Add consistency guarantees 2017-02-05 18:31:48 -08:00
Harshavardhana
533338bdeb all/windows: Be case in-sensitive about pattern matching. (#3682)
Resource strings and paths are case insensitive on windows
deployments but if user happens to use upper case instead of
lower case for certain configuration params like bucket
policies and bucket notification config. We might not honor
them which leads to a wrong behavior on windows.

This is windows only behavior, for all other platforms case
is still kept sensitive.
2017-02-03 23:27:50 -08:00
Anis Elleuch
b6ebf2aba8 server-mux: Simplify graceful shutdown behavior (#3681)
`*http.Server` is no more used, doing some cleanup.
2017-02-03 22:53:30 -08:00
Anis Elleuch
ed4fcb63f7 Require content-length in POST & Upload requests (#3671)
Avoid passing size = -1 to PutObject API by requiring content-length
header in POST request (as AWS S3 does) and in Upload web handler.
Post handler is modified to completely store multipart file to know
its size before sending it to PutObject().
2017-02-02 10:45:00 -08:00
Harshavardhana
4b4cb07fb6 Update browser assets with new changes. 2017-02-02 03:01:49 -08:00
Harshavardhana
fc446e0b81 docs: Add browser doc for running dev in custom port (#3675)
Fixes #3674
2017-02-01 13:12:13 -08:00
Anis Elleuch
f612a7dd85 madmin: Fix a typo in Locks duration query name (#3673) 2017-02-01 11:46:49 -08:00
Krishnan Parthasarathi
0472e5c1e1 Change query param name to duration in list/clear locks API (#3664)
Following is a sample list lock API request schematic,

  /?lock&bucket=mybucket&prefix=myprefix&duration=holdDuration
  x-minio-operation: list

The response would contain the list of locks held on mybucket matching
myprefix for a duration longer than holdDuration.
2017-02-01 11:17:30 -08:00
Harshavardhana
6a6c930f5b xl: Abort multipart upload should honor quorum properly. (#3670)
Current implementation didn't honor quorum properly and didn't
handle the errors generated properly. This patch addresses that
and also moves common code `cleanupMultipartUploads` into xl
specific private function.

Fixes #3665
2017-02-01 11:16:17 -08:00
Karthic Rao
35ca3e5d9b Fix unresponsiveness of doneCh due to Sleep call. (#3667) 2017-02-01 09:17:32 -08:00
Harshavardhana
a9ab01731f docs: Remove reference word from notification title 2017-01-31 18:17:31 -08:00
Harshavardhana
cd80e6df29 docs: Move the notifications into docs/bucket
Cleanup some formatting issues.
2017-01-31 18:07:39 -08:00
koolhead17
4dea4f3b89 docs: added event-notification back end doc. (#3510) 2017-01-31 17:04:36 -08:00
Harshavardhana
1b30a3be2b xl/utils: getPartSizeFromIdx should return error. (#3669) 2017-01-31 15:34:49 -08:00
Krishna Srinivas
f7f103725b browser: Make logo and "Minio Browser" text non-clickable. (#3668)
fixes #3601
2017-01-31 13:44:07 -08:00
Harshavardhana
77a192a7b5 Implement CopyObjectPart API (#3663)
This API is implemented to allow copying data from an
existing source object to an ongoing multipart operation

http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html

Fixes #3662
2017-01-31 09:38:34 -08:00
Harshavardhana
cb48517a78 Update browser assets with new changes. 2017-01-30 16:38:49 -08:00
Anis Elleuch
e9394dc22d xl PutObject: Split object into parts (#3651)
For faster time-to-first-byte when we try to download a big object
2017-01-30 15:44:42 -08:00
Krishna Srinivas
46743c7918 browser: Component attr names should not clash with redux state keys. (#3656)
fixes #270
2017-01-30 14:25:55 -08:00
Krishna Srinivas
b288eaddb3 xl: bit-rot algo was not set in get-object. (#3652)
fixes #3650
2017-01-30 14:25:28 -08:00
Krishnan Parthasarathi
2665aba555 Fail PutBucketPolicy if conditions are incompatible with actions. (#3659) 2017-01-30 09:20:16 -08:00
Krishnan Parthasarathi
9b6bcb30d9 Fix handling of StringNotEquals condition operator (#3660) 2017-01-30 09:18:10 -08:00
Harshavardhana
ac9ba13c19 build: Increase the travis build test timeout to 15mins 2017-01-30 01:39:53 -08:00
Krishnan Parthasarathi
864b8795aa heal: Should delete stale object parts before healing (#3649) 2017-01-30 00:45:56 -08:00
Bala FA
cc1575f944 fix: do port availability check only on macOS. (#3654)
On macOS, if a process already listens on 127.0.0.1:PORT, net.Listen() falls back
to IPv6 address ie minio will start listening on IPv6 address whereas another
(non-)minio process is listening on IPv4 of given port.
To avoid this error sutiation we check for port availability only for macOS.

Note: checkPortAvailability() tries to listen on given port and closes it.
It is possible to have a disconnected client in this tiny window of time.
2017-01-30 00:44:36 -08:00
Krishnan Parthasarathi
b408d0e87d Add aws:Referer condition key support. (#3641)
This change implements bucket policy enhancements required to restrict access based on HTTP referer.
See https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-4 for more information.

Fixes #3540
2017-01-29 19:45:11 -08:00
Nitish Tiwari
69b81af93e Fix Minio quick start guide URL (#3655) 2017-01-29 07:21:30 -08:00
Harshavardhana
cf558ff6d1 docs: Fix TLS doc to be docs.minio.io compatible 2017-01-28 22:46:29 -08:00
Harshavardhana
5af7bd6a01 docs: Fix a typo in TLS doc description 2017-01-28 22:41:52 -08:00
Harshavardhana
73f4f29110 docs: Restructure docs, move the files to their relevant location. (#3648)
Also combines windows TLS docs with single doc with Linux.
2017-01-28 19:45:30 -08:00
Harshavardhana
ccd949d8ca Add -v for race tests to avoid travis and appveyor timeouts. (#3647)
This is added so that CI's don't wrong fully timeout on
little longer running tests.
2017-01-27 12:31:02 -08:00
Anis Elleuch
e1bc99e4fe xl: Fix GET of an empty multiparted object (#3646)
GetObject returns unsatisfied range error when we try to download an object
uploaded using multipart mechanism.
2017-01-27 10:51:02 -08:00
Alex Ellis
02194ee3c6 Fix issue with webhook events not closing Response body, leaving connections open with remote server (#3645) 2017-01-27 09:50:08 -08:00
Harshavardhana
85f2b74cfd jwt: Cache the bcrypt password hash. (#3526)
Creds don't require secretKeyHash to be calculated
everytime, cache it instead and re-use.

This is an optimization for bcrypt.

Relevant results from the benchmark done locally, negative
value means improvement in this scenario.

```
benchmark                       old ns/op     new ns/op     delta
BenchmarkAuthenticateNode-4     160590992     80125647      -50.11%
BenchmarkAuthenticateWeb-4      160556692     80432144      -49.90%

benchmark                       old allocs     new allocs     delta
BenchmarkAuthenticateNode-4     87             75             -13.79%
BenchmarkAuthenticateWeb-4      87             75             -13.79%

benchmark                       old bytes     new bytes     delta
BenchmarkAuthenticateNode-4     15222         9785          -35.72%
BenchmarkAuthenticateWeb-4      15222         9785          -35.72%
```
2017-01-26 16:51:51 -08:00
Krishna Srinivas
152cdf1c05 fs: Move traceError() to lower functions where possible. (#3633) 2017-01-26 15:40:10 -08:00
Krishna Srinivas
17dd1c19df cleanup: refactor common code between FS and XL listDirFactory. (#3639) 2017-01-26 15:39:22 -08:00
Anis Elleuch
8e49a3d047 Simplify running cmd.Main() for external tests (#3636)
An external test that runs cmd.Main() has a difficulty to set cmd arguments
and MINIO_{ACCESS,SECRET}_KEY values, this commit changes a little the current
behavior in a way that helps external tests.
2017-01-26 15:22:41 -08:00
Krishna Srinivas
cccf77229d cleanup: Move code in *-multipart-common.go to *-multipart.go (#3638)
The code in *-multipart-common.go is not common anymore.
2017-01-26 12:51:12 -08:00
Harshavardhana
dafdc74605 fs: if fs.json is empty ignore it while reading metadata. (#3634)
This is needed so that we don't send wrong errors
on previously failed PutObject() which would have
left a stale `fs.json` entry.
2017-01-26 10:19:07 -08:00
Anis Elleuch
c753b5dfeb madmin: All APIs return ErrorResponse error type (#3632)
Avoid returning errors.New("Got HTTP Status") in all APIs
2017-01-25 16:01:29 -08:00
Krishna Srinivas
82373e3d50 fs: cleanup - do not cache size of metafiles (#3630)
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
2017-01-25 12:29:06 -08:00
Anis Elleuch
76b8abcd8e madmin: Fix args order in listObjectsHeal() (#3629)
The order of marker and delimiter and in listObjectsHeal() internal function
are switched. That will give wrong result in case of a non recursive objects
heal list.
2017-01-25 09:58:49 -08:00
Harshavardhana
dbb5408906 Add ARM64 build binary for README.md
Fixes #3626
2017-01-24 20:28:54 -08:00
Minio Trusted
29b49f9343 Bump to new release. 2017-01-24 19:18:54 -08:00
Minio Trusted
f8e4700a11 Merge remote-tracking branch 'origin/master' into release 2017-01-24 19:13:39 -08:00
Harshavardhana
18c9d49a32 Build browser with new changes and update yarn.lock 2017-01-24 19:11:37 -08:00
Minio Trusted
4c5c00c640 Bump to new release 2017-01-24 18:54:28 -08:00
Minio Trusted
0be44133d4 Merge remote-tracking branch 'origin/master' into release 2017-01-24 18:42:52 -08:00
Harshavardhana
d41dcb784b Move to blake2b-simd due to perf problems in golang.org/x/crypto
Ref https://github.com/golang/go/issues/18563
2017-01-24 18:07:34 -08:00
Krishna Srinivas
659d5aabd1 browser: Access to private paths redirects to login. (#3622) 2017-01-24 12:08:00 -08:00
Krishna Srinivas
b4343a28b7 browser: add yarn.lock and .gitignore files. (#3621) 2017-01-24 11:56:30 -08:00
Krishnan Parthasarathi
0e693e0284 Add dry-run query param for HealFormat API (#3618) 2017-01-24 08:11:05 -08:00
Anis Elleuch
fc880f9b23 admin: Enhance set credentials test (#3619)
Add more test cases and ignore access and secret keys set from the env
2017-01-24 08:08:36 -08:00
Krishna Srinivas
cead24b0f7 miniobrowser: Bring Minio browser source into minio repo. (#3617) 2017-01-23 18:07:22 -08:00
Krishna Srinivas
8489f22fe2 signature-v2: Use request.RequestURI for signature calculation. (#3616)
* signature-v2: Use request.RequestURI for signature calculation.

* Use splitStr instead of strings.Split
2017-01-23 17:01:44 -08:00
Anis Elleuch
fc6f804865 server-mux: Keep listening after Accept() err (#3613)
Accept() can return errors like: `too many open files`, no need to totally quit listening in this case.
2017-01-23 09:55:34 -08:00
Anis Elleuch
d1d89116f1 admin: Add version to service Status API response (#3605)
Add server's version field to service status API:

"version":{
	"version":"DEVELOPMENT.GOGET",
	"commitID":"DEVELOPMENT.GOGET"
}
2017-01-23 08:56:06 -08:00
Anis Elleuch
e1f64141a2 presign-v2: Compute signature on encoded URL path (#3612)
Encode the path of the passed presigned url before calculating the signature. This fixes
presigning objects whose names contain characters that are found encoded in urls.
2017-01-23 08:54:32 -08:00
Krishnan Parthasarathi
586058f079 Implement mgmt REST APIs to heal storage format. (#3604)
* Implement heal format REST API handler
* Implement admin peer rpc handler to re-initialize storage
* Implement HealFormat API in pkg/madmin
* Update pkg/madmin API.md to incl. HealFormat
* Added unit tests for ReInitDisks rpc handler and HealFormatHandler
2017-01-23 00:32:55 -08:00
Krishna Srinivas
4e926b292f vendor-update: Minio Browser (#3609) 2017-01-22 21:02:09 -08:00
Anis Elleuch
47358e3104 server-mux: Add tcp idle read timeout (#3607)
Avoid many idle client connections i.e client didn't send any data until a given stipulated amount of time. 

Default chosen here is 30 seconds.
2017-01-22 14:48:27 -08:00
Harshavardhana
3640c63289 server/mux: PeekProtocol() should return error and connection be closed. (#3608)
For TLS peekProtocol do not assume the incoming request to be a TLS
connection perform a handshake() instead and validate.

Also add some security related defaults to `tls.Config`.
2017-01-22 12:14:00 -08:00
Harshavardhana
51fa4f7fe3 Make PutObject a nop for an object which ends with "/" and size is '0' (#3603)
This helps majority of S3 compatible applications while not returning
an error upon directory create request.

Fixes #2965
2017-01-20 16:33:01 -08:00
Andrei Kopats
c3f7d1026f fs: start even if there are not enough free space (#3606) 2017-01-20 09:30:20 -08:00
Harshavardhana
80f1387877 Initialize peers properly for localhost. (#3600)
This introduced a regression.

Fixes #3594
2017-01-19 11:32:13 -08:00
Andreas Kohn
0674fa43ff Handle the region for GetBucketLocation and PutBucket properly (#3596)
This adjusts the code for these two handlers to match the logic in ListBucketHandler.

Fixes #3595
2017-01-19 11:31:51 -08:00
Harshavardhana
a17f1e875c server/mux: Close the connection even if buffer.Flush() returns error. (#3599)
It is possible that buf.Flush() might return an error, leading to a
potential leak in active sockets.
2017-01-19 11:19:57 -08:00
Jeffery Utter
9e1f1b50e0 Don't Check Available Inodes on NFS (#3598)
In some cases (such as with VirutualBox, this value gets hardcoded
to 1000, which is less than the required minimum of 10000.

Fixes #3592
2017-01-19 10:39:44 -08:00
Anis Elleuch
0715032598 heal: Add ListBucketsHeal object API (#3563)
ListBucketsHeal will list which buckets that need to be healed:
  * ListBucketsHeal() (buckets []BucketInfo, err error)
2017-01-19 09:34:18 -08:00
Harshavardhana
dfc2ef3004 storage/rpc: Remove network error restriction. (#3591)
This restriction has lots of side affects, since
we do not have a mechanism to clear states like
this it is better not to keep them.

Network errors are common and can occur with
simple cable removal etc. Since we already have
a retry mechanism this error count and stateful
nature can bring problems on a long running
cluster.
2017-01-18 12:55:57 -08:00
Harshavardhana
62f8343879 Add constants for commonly used values. (#3588)
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Anis Elleuch
f803bb4b3d admin: Add service Set Credentials API (#3580) 2017-01-17 14:25:59 -08:00
Andy Brown
20a65981bd add delete and options methods to CORS whitelist (#3589) 2017-01-17 14:20:05 -08:00
Harshavardhana
09b450d610 Fix fs tests to avoid deleting /usr to certain systems. 2017-01-17 14:05:07 -08:00
Krishnan Parthasarathi
c194b9f5f1 Implement mgmt REST APIs for heal subcommands (#3533)
The heal APIs supported in this change are,
- listing of objects to be healed.
- healing a bucket.
- healing an object.
2017-01-17 10:02:58 -08:00
Harshavardhana
98a6a2bcab obj: Return objectInfo for CompleteMultipartUpload(). (#3587)
This patch avoids doing GetObjectInfo() in similar way
how we did for PutOject().
2017-01-16 19:23:43 -08:00
Harshavardhana
1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Harshavardhana
a054c73e22 Add slack replace gitter (#3584) 2017-01-16 15:26:26 -08:00
Harshavardhana
b580ad24e2 server/http: Add missing keep alive for incoming tcp connections. (#3585)
This was seen reproducing a bug with @gowithplanb. Windows Cloud
Berry clients do not close their respective connections.
2017-01-16 03:38:06 -08:00
Harshavardhana
f8a3b1e164 Fix a bug in previous patch. 2017-01-16 01:48:34 -08:00
Harshavardhana
bf2b8879b7 config: Allow non-standard config dir to be configured with SSL. (#3583) 2017-01-15 16:53:01 -08:00
Harshavardhana
caecd75a2a Deprecate and remove service stop API. (#3578)
Fixes #3570
2017-01-14 14:48:52 -08:00
Anis Elleuch
2959c104b3 peer rpc: Fix typo in cluster credentials update (#3579)
Update credentials in cluster wasn't working due to a typo
2017-01-14 14:06:23 -08:00
Harshavardhana
50796e481d build: Add ARM binary builds for ARMv6 and Aarch64. (#3577) 2017-01-13 18:51:17 -08:00
Anis Elleuch
f64f8b03cb admin: Enhance locks list json response (#3573) 2017-01-13 14:25:34 -08:00
Harshavardhana
7b85756c64 notify/webhook: Handle webendpoints without port (#3568)
Fixes and issue initializing webhook notification

```
FATA[0000] Initializing object layer failed              cause=Unable to initialize event \
    notification. dial tcp: missing port in address requestb.in source=[server-main.go:448:serverMain()]
```
2017-01-12 20:08:00 -08:00
Alex
d6a327fbc5 Add notifications by webhook.
Add a new config entry moving to version 13.
```
		"webhook": {
			"1": {
				"enable": true,
				"address": "http://requestb.in/1i9al7m1"
			}
		}
```
2017-01-12 10:19:59 -08:00
Anis Elleuch
f24753812a nats: Add support of NATS.io Streaming server (#3494) 2017-01-11 16:41:05 -08:00
Harshavardhana
08b6cfb082 ssl: Set a global boolean to enable SSL across Minio (#3558)
We have been using `isSSL()` everywhere we can set
a global value once and re-use it again.
2017-01-11 13:59:51 -08:00
Krishna Srinivas
12a7a15daa browser: Allow anonymous browsing of readable buckets. (#3515) 2017-01-11 13:26:42 -08:00
Harshavardhana
7c6d77734a Fix vendor.json to remove minio/blake2b-simd ref 2017-01-11 00:26:00 -08:00
Harshavardhana
b0cfceb211 event: Enhance event message struct to provide origin server. (#3557)
`principalId` i.e user identity is kept as AccessKey in
accordance with S3 spec.

Additionally responseElements{} are added starting with

`x-amz-request-id` is a hexadecimal of the event time itself in nanosecs.
`x-minio-origin-server` - points to the server generating the event.

Fixes #3556
2017-01-10 16:43:48 -08:00
Harshavardhana
0563a9235a handlers: Handle crash if r.URL.Path is empty. (#3554)
URL paths can be empty and not have preceding separator,
we do not yet know the conditions this can happen inside
Go http server.

This patch is to ensure that we do not crash ourselves
under conditions where r.URL.Path may be empty.

Fixes #3553
2017-01-10 11:01:23 -08:00
Anis Elleuch
eb6d53d2f5 heal: Fix new entries computation in listDirHeal (#3551)
A crash was happening due to an incorrect interpreation of the return value of sort.SearchString()
2017-01-09 15:48:13 -08:00
Anis Elleuch
95d9e47353 Presign V2: Unescape non-std queries in urls (#3549)
A client sends escaped characters in values of some query parameters in a presign url.
This commit properly unescapes queires to fix signature calculation.
2017-01-09 14:22:20 -08:00
Harshavardhana
e1142e99f2 rpc/lock: Make sure to capitalize for proper marshalling. (#3544)
Distributed setup stopped working for certain
types of operations `6d10f4c19af6861e4de1b22ac20a3e5136f69d67`

This is a regression.

Fixes #3543
2017-01-08 20:37:53 -08:00
Harshavardhana
a091fe3ed6 docs: Fix caching docs to comply with current features. 2017-01-08 11:17:02 -08:00
koolhead17
f37f56ac15 docs: Added guide for distributed minio on Windows (#3507) 2017-01-07 18:35:12 -08:00
Karthic Rao
2f4a7483ea Test Function to reset globals. (#3538)
- Adding reset functions for important global variables.
- Using them in tests.
2017-01-07 11:27:01 -08:00
Harshavardhana
464f9d34d6 Remove all references to GO15VENDOREXPERIMENT 2017-01-06 18:42:32 -08:00
Harshavardhana
926c75d0b5 api: Set appropriate content-type for success/error responses. (#3537)
Golang HTTP client automatically detects content-type but
for S3 clients this content-type might be incorrect or
might misbehave.

For example:

```
Content-Type: text/xml; charset=utf-8
```

Should be
```
Content-Type: application/xml
```

Allow this to be set properly.
2017-01-06 00:37:00 -08:00
Krishnan Parthasarathi
c8f57133a4 Implement list, clear locks REST API w/ pkg/madmin support (#3491)
* Filter lock info based on bucket, prefix and time since lock was held
* Implement list and clear locks REST API
* madmin: Add list and clear locks API
* locks: Clear locks matching bucket, prefix, relTime.
* Gather lock information across nodes for both list and clear locks admin REST API.
* docs: Add lock API to management APIs
2017-01-03 23:39:22 -08:00
Harshavardhana
cae62ce543 browser: Handle proper login errors. (#3518)
Also additionally log the remote address.

Fixes #3514
2017-01-03 01:33:00 -08:00
Harshavardhana
7bbb532b4b Add a isErr function to check for errs.
DisksInfo() should handle collection of some
base errors as offlineDisks.
2017-01-02 10:52:43 -08:00
Bala.FA
6d10f4c19a Adopt dsync interface changes and major cleanup on RPC server/client.
* Rename GenericArgs to AuthRPCArgs
* Rename GenericReply to AuthRPCReply
* Remove authConfig.loginMethod and add authConfig.ServiceName
* Rename loginServer to AuthRPCServer
* Rename RPCLoginArgs to LoginRPCArgs
* Rename RPCLoginReply to LoginRPCReply
* Version and RequestTime are added to LoginRPCArgs and verified by
  server side, not client side.
* Fix data race in lockMaintainence loop.
2017-01-02 20:57:42 +05:30
Matthew Hall
cde6496172 checkdeps.sh: support unusual git version strings (for OS X et. al.) (#3512) 2016-12-31 19:57:33 -08:00
Harshavardhana
8562b22823 Fix delays and iterim fix for the partial fix in #3502 (#3511)
This patch uses a technique where in a retryable storage
before object layer initialization has a higher delay
and waits for longer period upto 4 times with time unit
of seconds.

And uses another set of configuration after the disks
have been formatted, i.e use a lower retry backoff rate
and retrying only once per 5 millisecond.

Network IO error count is reduced to a lower value i.e 256
before we reject the disk completely. This is done so that
combination of retry logic and total error count roughly
come to around 2.5secs which is when we basically take the
disk offline completely.

NOTE: This patch doesn't fix the issue of what if the disk
is completely dead and comes back again after the initialization.
Such a mutating state requires a change in our startup sequence
which will be done subsequently. This is an interim fix to alleviate
users from these issues.
2016-12-30 17:08:02 -08:00
Harshavardhana
dd68cdd802 Auto-reconnect for regular authRPC client. (#3506)
Implement a storage rpc specific rpc client,
which does not reconnect unnecessarily.

Instead reconnect is handled at a different
layer for storage alone.

Rest of the calls using AuthRPC automatically
reconnect, i.e upon an error equal to `rpc.ErrShutdown`
they dial again and call the requested method again.
2016-12-29 19:42:02 -08:00
Harshavardhana
41cf580bb1 Improve reconnection logic, allow jitters. (#3502)
Attempt a reconnect also if disk not found.

This is needed since any network operation error
is converted to disk not found but we also need
to make sure if disk is really not available. 

Additionally we also need to retry more than
once because the server might be in startup
sequence which would render other servers to
wrongly think that the server is offline.
2016-12-29 03:13:51 -08:00
Dee Koder
00baec1746 syntax highlighting : Remove sh from code block which incorrectly highlights shell commands. (#3504) 2016-12-28 16:07:05 -08:00
Harshavardhana
f9025652df Support building specific architectures. 2016-12-27 17:52:24 -08:00
Bala FA
ee0172dfe4 Have simpler JWT authentication. (#3501) 2016-12-27 08:28:10 -08:00
Harshavardhana
69559aa101 objAPI: Implement CopyObject API. (#3487)
This is written so that to simplify our handler code
and provide a way to only update metadata instead of
the data when source and destination in CopyObject
request are same.

Fixes #3316
2016-12-26 16:29:26 -08:00
Krishnan Parthasarathi
36fd317eb2 Clean up lock-instrumentation and improve comments (#3499)
- Add a lockStat type to group counters
- Remove unnecessary helper functions
- Fix stats computation on force unlock
- Removed unnecessary checks and cleaned up comments
2016-12-26 10:29:55 -08:00
Bala FA
e8ce3b64ed Generate and use access/secret keys properly (#3498) 2016-12-26 10:21:23 -08:00
Andreas Auernhammer
6ee27daac1 fix blake2b tests on non-amd64 machines (#3496)
Fix the TestHashes Test for non-amd64 machines
2016-12-25 17:03:08 -08:00
koolhead17
cdc6c2d578 docs: Removed unmaintained docs. (#3493)
* docs: Removed unmaintained docs.

* docs: removed associated screenshots with the older docs.
2016-12-23 22:06:01 -08:00
Harshavardhana
855ef4f1aa Fix typo in erasure-utils.go 2016-12-22 10:47:10 -08:00
Harshavardhana
5878fcc086 bit-rot: Default to sha256 on ARM64. (#3488)
This is to utilize an optimized version of
sha256 checksum which @fwessels implemented.

blake2b lacks such optimizations on ARM platform,
this can provide us significant boost in performance.

blake2b on ARM64 as expected would be slower.
```
BenchmarkSize1K-4           	   30000	     44015 ns/op	  23.26 MB/s
BenchmarkSize8K-4           	    5000	    335448 ns/op	  24.42 MB/s
BenchmarkSize32K-4          	    1000	   1333960 ns/op	  24.56 MB/s
BenchmarkSize128K-4         	     300	   5328286 ns/op	  24.60 MB/s
```

sha256 on ARM64 is faster by orders of magnitude giving close to
AVX performance of blake2b.
```
BenchmarkHash8Bytes-4	 1000000	      1446 ns/op	   5.53 MB/s
BenchmarkHash1K-4    	  500000	      3229 ns/op	 317.12 MB/s
BenchmarkHash8K-4    	  100000	     14430 ns/op	 567.69 MB/s
BenchmarkHash1M-4    	    1000	   1640126 ns/op	 639.33 MB/s
```
2016-12-22 08:25:03 -08:00
Andreas Auernhammer
1ac36a95aa replace blake2b implementation (#3481)
* replace blake2b implementation
replace the blake2b-simd with the golang/x/crypto implementation

```
name        old time/op    new time/op     delta
Size64-8       715ns ±13%      614ns ± 3%    ~     (p=0.084 n=6+6)
Size128-8      612ns ± 5%      634ns ± 8%    ~     (p=0.084 n=6+6)
Size1K-8      2.18µs ± 5%     2.09µs ± 7%    ~     (p=0.084 n=6+6)
Size8K-8      13.1µs ± 2%     13.4µs ± 3%    ~     (p=0.084 n=6+6)
Size32K-8     48.5µs ± 1%     49.5µs ± 3%    ~     (p=0.775 n=6+6)
Size128K-8     199µs ± 0%      198µs ± 3%    ~     (p=0.468 n=6+6)

name        old speed      new speed       delta
Size64-8    92.6MB/s ±11%  104.2MB/s ± 3%    ~     (p=0.139 n=6+6)
Size128-8    208MB/s ± 6%    202MB/s ± 8%    ~     (p=0.102 n=6+6)
Size1K-8     466MB/s ± 7%    492MB/s ± 7%    ~     (p=0.139 n=6+6)
Size8K-8     621MB/s ± 2%    610MB/s ± 3%    ~     (p=0.102 n=6+6)
Size32K-8    672MB/s ± 2%    669MB/s ± 1%    ~     (p=0.818 n=6+6)
Size128K-8   657MB/s ± 1%    672MB/s ± 0%  +2.28%  (p=0.002 n=6+6)

name        old time/op   new time/op   delta
Size64-4      334ns ± 1%    243ns ± 0%  -27.14%  (p=0.029 n=4+4)
Size128-4     296ns ± 1%    242ns ± 0%  -18.21%  (p=0.029 n=4+4)
Size1K-4     1.44µs ± 0%   1.28µs ± 0%  -10.83%  (p=0.029 n=4+4)
Size8K-4     10.0µs ± 0%    9.4µs ± 0%   -6.23%  (p=0.029 n=4+4)
Size32K-4    39.8µs ± 1%   37.3µs ± 0%   -6.31%  (p=0.029 n=4+4)
Size128K-4    162µs ± 3%    149µs ± 0%   -7.72%  (p=0.029 n=4+4)

name        old speed     new speed     delta
Size64-4    192MB/s ± 1%  263MB/s ± 0%  +37.24%  (p=0.029 n=4+4)
Size128-4   431MB/s ± 0%  526MB/s ± 0%  +22.04%  (p=0.029 n=4+4)
Size1K-4    713MB/s ± 0%  800MB/s ± 0%  +12.17%  (p=0.029 n=4+4)
Size8K-4    815MB/s ± 0%  869MB/s ± 0%   +6.64%  (p=0.029 n=4+4)
Size32K-4   823MB/s ± 1%  878MB/s ± 0%   +6.72%  (p=0.029 n=4+4)
Size128K-4  810MB/s ± 3%  877MB/s ± 0%   +8.23%  (p=0.029 n=4+4)
```
See: https://go-review.googlesource.com/#/c/34319/
2016-12-21 14:20:01 -08:00
Harshavardhana
15b4c49621 fs/xl: Simplify bucket metadata reading. (#3486)
ObjectLayer GetObject() now returns the entire object
if starting offset is 0 and length is negative. This
also allows to simplify handler layer code where
we always had to use GetObjectInfo() before proceeding
to read bucket metadata files examples `policy.json`.

This also reduces one additional call overhead.
2016-12-21 11:29:32 -08:00
Harshavardhana
f57f773189 admin: Add missing madmin examples and API docs. (#3483) 2016-12-20 18:49:48 -08:00
Harshavardhana
e7b4e4e105 admin: ServiceStatus() shouldn't have to write double http headers. (#3484)
Fixes #3482
2016-12-20 18:05:25 -08:00
Harshavardhana
1d134c1a94 admin: Fix wrong vendorized location for s3signer and s3utils. 2016-12-20 15:18:37 -08:00
Anis Elleuch
329a910b86 Admin Lib: Implement Service API (#3426)
Three APIs were added to control a minio server

* NewAdminClient()
* ServiceStop()
* ServiceRestart()
* ServiceStatus()
2016-12-20 14:45:17 -08:00
Harshavardhana
4309727354 docs: Add docs for minio limitations. (#3477) 2016-12-20 14:42:13 -08:00
Anis Elleuch
ef3319a49d Fix: Typo in non canonicalized header extraction (#3480)
Extracting metadata from headers was doing wrong when Headers are not well canonicalized, fixing typo.
2016-12-20 11:59:08 -08:00
Anis Elleuch
d8e4d3c9c8 POSTForm: Return http 303 if redirect is specified (#3479)
success_action_redirect in the sent Form means that the server needs to return 303 in addition to a well specific redirection url, this commit adds this feature
2016-12-20 09:32:17 -08:00
Harshavardhana
faa6b1e925 vendorize deps for snappy, blake2b and sha256 (#3476)
Bring in new optimization and portability changes.

Fixes https://github.com/minio/minio-go/issues/578
2016-12-19 19:32:55 -08:00
Krishnan Parthasarathi
85c6bb9809 server: Sort disk arguments for consistent ordering (#3469)
This is important in a distributed setup, where the server hosting the
first disk formats a fresh setup. Sorting ensures that all servers
arrive at the same 'first' server.

Note: This change doesn't protect against different disk arguments
with some disks being same across servers.
2016-12-19 17:04:31 -08:00
Anis Elleuch
5404dddcea PostForm: Save supported headers in obj metadata (#3474)
Supported Headers like Content-Type, Cache-Control, Content-Encoding, X-Amz-* , etc.. are now saved in object metadata
2016-12-19 16:14:04 -08:00
Anis Elleuch
4692fdbb8f PostForm: Follow success_action_status requirement (#3467)
S3 spec requires that Post Object response depends on the passed success_action_status. This commit implements that requirement.
2016-12-18 13:39:56 -08:00
Bala FA
1875a47495 rpcclient: fix leaky connection (#3471)
Previously, more than one goroutine calls RPCClient.dial(), each
goroutine gets a new rpc.Client but only one such client is stored
into RPCClient object.  This leads to leaky connection at the server
side.  This is fixed by taking lock at top of dial() and release on
return.
2016-12-17 18:17:40 -08:00
Harshavardhana
9c9f390350 fs: validate filesystem path argument properly. (#3470)
FS should fail for invalid paths like

 - file:///
 - ftp://
 - http://
2016-12-17 13:43:26 -08:00
Bala FA
1b2b16998f Remove regexp usage (#3456)
This patch removes regexp usage in cacheControlHandler.ServeHTTP() and
server-main.checkEndpointsSyntax()
2016-12-17 11:00:16 -08:00
koolhead17
7a17b2a585 Browser: Added character limit for Bucket Name. (#3454) 2016-12-16 09:59:37 -08:00
Aditya Manthramurthy
f7766b49aa Amend ZFS line in README - to make it about Minio on FreeNAS (#3461) 2016-12-16 09:59:02 -08:00
Krishnan Parthasarathi
b2f920a868 Add service API handler stubs for status, stop and restart (#3417) 2016-12-15 22:26:15 -08:00
Anis Elleuch
8ceb969445 tests: Use testTmpDir var to specify tmp directory (#3459)
To be able to specify the directory where tests will be done.  This way,
it will be easier to run Minio tests on a mounted directory like NFS, ..
2016-12-15 22:25:05 -08:00
Harshavardhana
0db484c8f6 signv2: Do not use path encoding for query values. (#3458)
Use query unescape before comparing signature.
2016-12-15 14:56:18 -08:00
dcharbonnier
5c481fbf6e Fix Minio Quickstart Guide link (#3457) 2016-12-15 11:59:16 -08:00
Aditya Manthramurthy
8e6e9301ce Add support for Kafka as a notifications target (#2869) (#3439) 2016-12-15 08:23:48 -08:00
Harshavardhana
664ff063a1 server: checkEndpoints syntax properly. (#3451) 2016-12-14 20:42:19 -08:00
koolhead17
d9fd6f9a96 docs: Removed $/# from code block to make code copy easier. (#3448) 2016-12-14 17:45:47 -08:00
Harshavardhana
b28ff50126 lock/server: Check if the lock server itself is skewed back. (#3447) 2016-12-14 08:02:32 -08:00
Karthic Rao
3fe2d77b70 Adding functions for resetting globals. (#3421) 2016-12-13 11:51:48 -08:00
Krishnan Parthasarathi
ab49498fc3 server: Exit gracefully if no endpoint is local to it. (#3442) 2016-12-13 11:18:31 -08:00
Minio Trusted
46fdd70114 Fix docker file and avoid creating README.md 2016-12-13 09:31:19 -08:00
Minio Trusted
817e763416 Bump to new release. 2016-12-13 09:24:36 -08:00
Minio Trusted
0abab5305f Merge branch 'master' into release 2016-12-13 09:22:41 -08:00
Krishnan Parthasarathi
29d72b84c0 loginServer should be a member of lockServer (#3441)
Add a unit test to catch a missing LoginHandler method in lockServer.
2016-12-13 08:01:47 -08:00
Anand Babu (AB) Periasamy
f53fcdf10e keep make messages consistent (#3437) 2016-12-13 00:26:45 -08:00
Krishna Srinivas
8e665105b1 Presign-v4: Allow requests that were signed slightly ahead of the current time. (#3435) 2016-12-12 16:32:00 -08:00
Minio Trusted
e6e855a7cf Bump to new release. 2016-12-12 15:49:31 -08:00
Minio Trusted
6fca1d164c Merge branch 'master' into release 2016-12-12 15:48:03 -08:00
Harshavardhana
2062add05f fs/posix: On windows use helpers and init format.json properly. (#3434)
Fixes #3433
2016-12-12 15:43:41 -08:00
Minio Trusted
9b76eba37e Bump to new release RELEASE.2016-12-12T18-35-43Z 2016-12-12 10:51:35 -08:00
Minio Trusted
03714d4809 Merge branch 'master' into release 2016-12-12 10:50:37 -08:00
Krishnan Parthasarathi
6b4e6bcebf Move LoginHandler into LoginServer which others embed (#3431)
* Move LoginHandler into LoginServer which others embed

* Add unit tests for loginServer
2016-12-12 08:11:23 -08:00
Harshavardhana
2d6f8153fa format: Check properly for disks in valid formats. (#3427)
There was an error in how we validated disk formats,
if one of the disk was formatted and was formatted with
FS would cause confusion and object layer would never
initialize essentially go into an infinite loop.

Validate pre-emptively and also check for FS format
properly.
2016-12-11 15:18:55 -08:00
Anis Elleuch
5c10f4adf0 presign v2: include resp headers in signature calc (#3428)
Include response headers when presigning an url using signature v2 algorithm
2016-12-11 14:32:25 -08:00
Harshavardhana
4daa0d2cee lock: Moving locking to handler layer. (#3381)
This is implemented so that the issues like in the
following flow don't affect the behavior of operation.

```
GetObjectInfo()
.... --> Time window for mutation (no lock held)
.... --> Time window for mutation (no lock held)
GetObject()
```

This happens when two simultaneous uploads are made
to the same object the object has returned wrong
info to the client.

Another classic example is "CopyObject" API itself
which reads from a source object and copies to
destination object.

Fixes #3370
Fixes #2912
2016-12-10 16:15:12 -08:00
Harshavardhana
cd0f350c02 env: Bring back MINIO_BROWSER env. (#3423)
Set MINIO_BROWSER=off to disable web browser completely.

Fixes #3422
2016-12-10 00:42:22 -08:00
Krishna Srinivas
ac554bf663 FS/Multipart: Fix race between PutObjectPart and Complete/Abort multi… (#3419)
FS/Multipart: Fix race between PutObjectPart and Complete/Abort multipart. close(timeoutCh) on complete/abort so that a racing PutObjectPart does not leave a dangling go-routine.

Fixes #3351
2016-12-09 16:10:18 -08:00
Harshavardhana
b363709c11 caching: Optimize memory allocations. (#3405)
This change brings in changes at multiple places

 - Reuse buffers at almost all locations ranging
   from rpc, fs, xl, checksum etc.
 - Change caching behavior to disable itself
   under low memory conditions i.e < 8GB of RAM.
 - Only objects cached are of size 1/10th the size
   of the cache for example if 4GB is the cache size
   the maximum object size which will be cached
   is going to be 400MB. This change is an
   optimization to cache more objects rather
   than few larger objects.
 - If object cache is enabled default GC
   percent has been reduced to 20% in lieu
   with newly found behavior of GC. If the cache
   utilization reaches 75% of the maximum value
   GC percent is reduced to 10% to make GC
   more aggressive.
 - Do not use *bytes.Buffer* due to its growth
   requirements. For every allocation *bytes.Buffer*
   allocates an additional buffer for its internal
   purposes. This is undesirable for us, so
   implemented a new cappedWriter which is capped to a
   desired size, beyond this all writes rejected.

Possible fix for #3403.
2016-12-08 20:35:07 -08:00
Anis Elleuch
410b579e87 startup: Show elapsed time in disks format process (#3413) 2016-12-07 10:22:00 -08:00
Karthic Rao
7b7c0bba58 Use a non member mutex lock for serverConfig access. (#3411)
- This is to ensure that the any new config references made to the
  serverConfig is also backed by a mutex lock.
- Otherwise any new config assigment will also replace the member mutex
  which is currently used for safe access.
2016-12-07 03:41:54 -08:00
Anis Elleuch
0cef971832 Fix max cache size calculation when system RAM is inferior to the default cache size (#3410) 2016-12-06 16:09:26 -08:00
Anis Elleuch
5c9a95df32 srv-mux: do not print peek protocol EOF err msg (#3402)
EOF err message in Peek Protocol is shown when a client closes the
connection in the middle of peek protocol, this commit hides it since it
doesn't make sense to show it
2016-12-05 14:49:32 -08:00
Anis Elleuch
3b455d6137 tests: Add tests for xl-v1-list-objects-heal (#3399) 2016-12-05 09:40:33 -08:00
Anis Elleuch
b2a0e5754b bucket-handlers: More tests for post form handler (#3392) 2016-12-04 12:23:19 -08:00
Anis Elleuch
63d9bb626a postform: fix check when ${filename} is provided (#3391)
Checking key condition when ${filename} is provided wasn't working well,
this patch fixes the wrong behavior
2016-12-04 10:30:52 -08:00
Anis Elleuch
372da5eaf5 tests: Enhance checkPostPolicy() coverage (#3389) 2016-12-03 12:41:07 -08:00
Harshavardhana
cf17fc7774 fs: PutObject create 0byte objects properly. (#3387)
Current code always appends to a file only if 1byte or
more was sent on the wire was affecting both PutObject
and PutObjectPart uploads.

This patch fixes such a situation and resolves #3385
2016-12-03 11:53:12 -08:00
Krishnan Parthasarathi
67509453d3 FS: sync abortMultipart cleanup and bg append (#3388)
backgroundAppend type's abort method should wait for appendParts to finish
writing ongoing appending of parts in the background before cleaning up
the part files.
2016-12-02 23:33:06 -08:00
Harshavardhana
d31f256020 Fail on lint errors during CI build. 2016-12-02 18:08:12 -08:00
Harshavardhana
d67f47927c api: Fix the formatting issues in last patch. 2016-12-02 17:39:21 -08:00
Anis Elleuch
85bb5870a9 Post Policy Form: exhaustive post policy check (#3386)
Add support of all conditions check described in
http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html
and simplify a little bit the existing code
2016-12-02 17:00:33 -08:00
Harshavardhana
4a9b205a15 docs: Add missing comments for exported functions. 2016-12-02 14:39:23 -08:00
koolhead17
f3b346cbb3 docs: removed the Edge tag reference. (#3366) 2016-12-02 10:47:18 -08:00
Harshavardhana
ff4ce0ee14 fs/xl: Combine input checks into re-usable functions. (#3383)
Repeated code around both object layers are moved
and combined into simple re-usable functions.
2016-12-01 23:15:17 -08:00
Anis Elleuch
918924796f fs: Enable shutdown test with faulty disks (#3380) 2016-12-01 13:59:06 -08:00
Krishnan Parthasarathi
feb6685359 posix: Use preparePath only for paths used with syscall or os functions (#3377) 2016-11-30 20:56:15 -08:00
Bala FA
0d59ea1e94 postpolicy: handle Amazon S3 compatible content-length-range condition (#3376)
Previously minio server expects content-length-range values as integer
in JSON.  However Amazon S3 handles content-length-range values as
integer and strings.

This patch adds support for string values.
2016-11-30 18:30:59 -08:00
Krishna Srinivas
38edd94282 ListBuckets: Allow listBuckets request to be signed with region configured in config.json (#3374)
Fixes #3373
2016-11-30 13:55:56 -08:00
Krishna Srinivas
8021061bd8 Implement BucketUpdater interface to call BucketMetaState methods. (#3375) 2016-11-30 13:37:38 -08:00
Anand Babu (AB) Periasamy
bc9509bc8a edits on Limits 2016-11-30 01:58:31 -08:00
Krishna Srinivas
e3b4910b66 FS/CompleteMultipart: lock the namespace before renaming the appended tmp file. (#3371) 2016-11-29 23:26:36 -08:00
Harshavardhana
d056f19d07 api: Allow reconnection of policy/notification rpc clients. (#3368)
Since we moved out reconnection logic from net-rpc-client.go
we should do it from the top-layer properly and bring back
the code to reconnect properly in-case the connection is lost.
2016-11-29 22:39:32 -08:00
Harshavardhana
834007728c fs: Do not print redundant md5Sum response header. (#3369)
For both GET and HEAD requests.
2016-11-29 16:47:01 -08:00
Krishnan Parthasarathi
a609a4126c Vendorize recent fixes in dsync (#3365)
* Update 'i' only if lock grant read from buffered channel
2016-11-28 23:03:46 -08:00
Krishna Srinivas
bcd1a2308b FS/Shutdown: cleanup and delete .minio.sys during Shutdown() (#3360) 2016-11-28 22:54:48 -08:00
koolhead17
694bad434c docs: Modified the Docker doc to reflect distributed release image & (#3362)
modification.
2016-11-28 12:45:35 -08:00
Anis Elleuch
01f625824a args: Honor config-dir & quiet wherever they are (#3356)
setGlobalsFromContext() is added to set global variables after parsing
command line arguments. Thus, global flags will be honored wherever
they are placed in minio command.
2016-11-28 12:15:36 -08:00
Bala FA
9ccfb70104 Minor cleanup. (#3361) 2016-11-28 12:14:24 -08:00
Harshavardhana
201a20ac02 handlers: Handle re-direction properly for S3 requests. (#3355)
Make sure all S3 signature requests are not re-directed
to `/minio`. This should be only done for JWT and some
Anonymous requests.

This also fixes a bug found from https://github.com/bji/libs3

```
$ s3 -u list

ERROR: XmlParseFailure
```

Now after this fix shows proper output
```
$ s3 -u list
                         Bucket                                 Created
--------------------------------------------------------  --------------------
andoria                                                   2016-11-27T08:19:06Z
```
2016-11-27 16:30:46 -08:00
Krishna Srinivas
f3322e94c8 FS: Skip creating fs.json for objects created by minio (ex. policy.json) (#3353) 2016-11-27 11:33:08 -08:00
Minio Trusted
3a73ea3f54 Bump to new release 2016-11-25 18:38:44 -08:00
Harshavardhana
9625629fc7 Merge branch 'master' into release 2016-11-25 18:23:36 -08:00
Harshavardhana
46a6fde813 xl/fs: Fix initializing meta volume bug. 2016-11-25 18:17:53 -08:00
Anis Elleuch
fd1f09a66c log: Enable loggers just after configuration load (#3348)
It would make sense to enable logger just after config initialisation.
That way, errorIf() and fatalIf() will be usable and can catch error
like invalid access and key errors.
2016-11-25 10:39:00 -08:00
Remco Verhoef
752ed7915b Add windows services docs with nssm instead of sc. 2016-11-24 16:53:10 -08:00
Bala FA
d3064e40b3 isDocker() logs error than fatal error. (#3347) 2016-11-24 16:06:49 -08:00
Bala FA
39f9324616 Remove uncessary err != nil check. (#3346) 2016-11-24 15:22:33 -08:00
Aditya Manthramurthy
a822b8e782 docs: Fix command sample to be copy-pastable (#3345) 2016-11-24 11:38:19 -08:00
Harshavardhana
a6922f94ee docs: Fix more formatting issues. 2016-11-23 20:25:05 -08:00
Harshavardhana
b4d57bf4c4 docs: Fix some formatting issues for rendering 2016-11-23 20:20:12 -08:00
Bala FA
0f2e493c9a Use isErrIgnored() function wherever applicable. (#3343) 2016-11-23 20:05:04 -08:00
Nitish Tiwari
4ef2d8940c Rename quick-start-guide.md to README.md (#3344) 2016-11-23 20:04:29 -08:00
Nitish Tiwari
96099a1e97 Added distributed Minio architecture images and quick start guide (#3342) 2016-11-23 19:30:23 -08:00
Minio Trusted
8507ce2a87 Bump to new release. 2016-11-23 18:19:40 -08:00
Harshavardhana
0e87f29de9 Disable heal message printing, comment it out as todo. 2016-11-23 17:54:29 -08:00
Harshavardhana
dd74e5a809 Revert "init: Honor config-dir flag when it is passed as global or local flag (#3337)"
This reverts commit e2ef95af7d.

This is reverted since the previous patch caused crashes.
2016-11-23 17:31:36 -08:00
Harshavardhana
12c1abed98 Vendorize with bug fixes from minio browser. (#3341)
This patch brings in changes from miniobrowser repo.

- Bucket policy UI and functionality fixes by @krishnasrinivas
- Bucket policy implementation by @balamurugana
- UI changes and new functionality changing password etc. @rushenn
- UI and new functionality for sharing URLs, deleting files
  @rushenn and @krishnasrinivas.
- Other misc fixes by @vadmeste @brendanashworth
2016-11-23 17:31:11 -08:00
Anis Elleuch
e2ef95af7d init: Honor config-dir flag when it is passed as global or local flag (#3337)
setGlobalsFromContext() is added to sets global variable after parsing
command line arguments.
2016-11-23 17:13:40 -08:00
Harshavardhana
d711ff454e logs: Do not log common successful errors. (#3340)
Errors like `BucketNotFound`, `BucketExists` shouldn't be logged.

Fixes #3229
2016-11-23 16:36:26 -08:00
Anis Elleuch
c667d20dfc config-migrate: Fix buggy continuous re-migration of v9 to v10 config (#3338) 2016-11-23 15:53:55 -08:00
Harshavardhana
6efee2072d objectLayer: Check for format.json in a wrapped disk. (#3311)
This is needed to validate if the `format.json` indeed exists
when a fresh node is brought online.

This wrapped implementation also connects to the remote node
by attempting a re-login. Subsequently after a successful
connect `format.json` is validated as well.

Fixes #3207
2016-11-23 15:48:10 -08:00
Harshavardhana
7a5bbf7a2e Revert "Vendor update for dsync, fixing major go routine leak issue. (#3308)"
This reverts commit 273228fafa.
2016-11-23 15:17:41 -08:00
Anis Elleuch
14cb3645a3 config/logger: remove syslogger and upgrade to config v10 which eliminates syslog config (#3336) 2016-11-23 15:00:53 -08:00
Krishna Srinivas
f4f512fedd FS/multipart: Bug fix related to part path. Hold lock on part while appending. (#3335) 2016-11-23 12:50:09 -08:00
Nitish Tiwari
1983925dcf Added distributed Minio architecture files. (#3330) 2016-11-23 11:36:52 -08:00
Anis Elleuch
22c98d3fa2 logger: Disassociate shared log config between console, file and syslog (#3333)
logurs is not helping us to set different log formats (json/text) to
different loggers. Now, we create different logurs instances and call
them in errorIf and fatalIf
2016-11-23 11:35:04 -08:00
Krishna Srinivas
01ae5bb39c FS/multipart: Fix append-parts to use minioMetaTmpBucket. (#3304) 2016-11-23 03:04:04 -08:00
koolhead17
11faf3f16d docs: Modified FreeBSD guide with official how to install golang link, (#3293) 2016-11-23 02:35:19 -08:00
Remco Verhoef
d3df6c711a How to run Minio as a service on Windows (#3327) 2016-11-23 02:27:54 -08:00
Bala FA
ed6e781679 globals: make read only variables as constants. (#3326) 2016-11-22 20:13:20 -08:00
Bala FA
baf1c1638d server: set maximum allowed request body. (#3324)
This patch sets the value as 5GiB + 64MiB.  5GiB is the maximum
allowed object size and 64MiB is for form data fields and headers.
2016-11-22 19:58:51 -08:00
Bala FA
825000bc34 Use humanize constants for KiB, MiB and GiB units. (#3322) 2016-11-22 18:18:22 -08:00
Bala FA
c1ebcbcda2 Remove usused code. (#3321) 2016-11-22 17:44:18 -08:00
Anis Elleuch
41a3a9e402 server: forbid zero port in address flag since it confuses clients and (#3318) 2016-11-22 17:01:15 -08:00
Bala FA
1d4ac4b084 Rename getUUID() into mustGetUUID() (#3320)
In case of UUID generation failure mustGetUUID() will panic than
infinitely trying in for loop.
2016-11-22 16:52:37 -08:00
Bala FA
71b357e4f2 Remove uploadIDChange structure. (#3309)
addUploadID() and removeUploadID() are wrappers to updateUploadJSON()
which is called with respective arguments.
2016-11-22 15:29:39 -08:00
Anis Elleuch
339c9019b9 Protect multipart directory from removing when it is empty (#3315) 2016-11-22 13:15:06 -08:00
Harshavardhana
dd93f808c8 web: Add more data for jsonrpc responses. (#3296)
This change adds more richer error response
for JSON-RPC by interpreting object layer
errors to corresponding meaningful errors
for the web browser.

```go
&json2.Error{
   Message: "Bucket Name Invalid, Only lowercase letters, full stops, and numbers are allowed.",
}
```

Additionally this patch also allows PresignedGetObject()
to take expiry parameter to have variable expiry.
2016-11-22 11:12:38 -08:00
Anis Elleuch
4098025c11 Remove XL multipart tmp files when the latter is canceled (#3214)
XL multipart fails to remove tmp files when an error occurs during upload, this case covers the scenario where an upload is canceled manually by the client in the middle of job.
2016-11-21 16:34:57 -08:00
Bala FA
bef0a50bc1 Cleanup and fixes (#3273)
* newRequestID() (previously generateUploadID()) returns string than byte array.
* Remove unclear comments and added appropriate comments.
* SHA-256, MD5 Hash functions return Hex/Base64 encoded string than byte array.
* Remove duplicate MD5 hasher functions.
* Rename listObjectsValidateArgs() into validateListObjectsArgs()
* Remove repeated auth check code in all bucket request handlers.
* Remove abbreviated names in bucket-metadata
* Avoid nested if in bucketPolicyMatchStatement()
* Use ioutil.ReadFile() instead of os.Open() and ioutil.ReadAll()
* Set crossDomainXML as constant.
2016-11-21 13:51:05 -08:00
Anis Elleuch
71ada9d6f8 commitXLMetadata() expects src and dst buckets in its arguments (#3307) 2016-11-21 12:45:02 -08:00
tibbes
33c022fcec Fix checkdeps.sh on Mac (#3306)
Update the check_minimum_version function to use numeric comparison (not
string comparison) on components of version numbers. Fixes the following
output:

```
$ make
Checking deps:
ERROR
OSX version '10.11.6' not supported.
Minimum supported version: 10.8
make: *** [checks] Error 1
```
2016-11-21 12:25:46 -08:00
Karthic Rao
273228fafa Vendor update for dsync, fixing major go routine leak issue. (#3308) 2016-11-21 10:47:48 -08:00
Harshavardhana
aa98702908 api: Handle content-length-range policy properly. (#3297)
content-length-range policy in postPolicy API was
not working properly handle it. The reflection
strategy used has changed in recent version of Go.
Any free form interface{} of any integer is treated
as `float64` this caused a bug where content-length-range
parsing failed to provide any value.

Fixes #3295
2016-11-21 04:15:26 -08:00
Harshavardhana
5197649081 utils: reduceErrs returns and validates quorum errors. (#3300)
This is needed as explained by @krisis

Lets say we have following errors.

```
[]error{nil, errFileNotFound, errDiskAccessDenied, errDiskAccesDenied}
```

Since the last two errors are filtered, the maximum is nil,
depending on map order.

Let's say we get nil from reduceErr. Clearly at this point
we don't have quorum nodes agreeing about the data and since
GetObject only requires N/2 (Read quorum) and isDiskQuorum
would have returned true. This is problematic and can lead to
undersiable consequences.

Fixes #3298
2016-11-21 01:47:26 -08:00
Harshavardhana
066f64d34a bootup: MetaVolume init should use isErrIngored helper. (#3303) 2016-11-21 01:46:55 -08:00
Krishna Srinivas
afa4c7c3ef fs/multipart: Append multipart parts in a proper Go routine in background. (#3282) 2016-11-20 23:42:53 -08:00
Krishna Srinivas
38537c7df2 Print line numbers to give more info on the failed tests in ExecObjectLayerAPIAnonTest() (#3302) 2016-11-20 23:41:39 -08:00
Krishnan Parthasarathi
eed9ab0464 XL: pickValidXLMeta should return error instead of panic'ing (#3277) 2016-11-20 20:56:44 -08:00
Harshavardhana
0b9f0d14a1 auth/rpc: Take remote disk offline after maximum allowed attempts. (#3288)
Disks when are offline for a long period of time, we should
ignore the disk after trying Login upto 5 times.

This is to reduce the network chattiness, this also reduces
the overall time spent on `net.Dial`.

Fixes #3286
2016-11-20 16:57:12 -08:00
Anis Elleuch
ffbee70e04 Avoid removing 'tmp' directory inside '.minio.sys' (#3294) 2016-11-20 14:25:43 -08:00
Harshavardhana
2c3a2241e7 update: Change update notifier for new style banner. (#3289)
For binary releases and operating systems it would be

All operating systems.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 30 minutes old                           ┃
┃ Update: https://dl.minio.io/server/minio/release/linux-amd64/minio ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```

On docker.
```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Minio is 25 days 12 hours 32 minutes old ┃
┃ Update: docker pull minio/minio          ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
```
2016-11-19 23:20:13 -08:00
Harshavardhana
1c47365445 xl/bootup: Upon bootup handle errors loading bucket and event configs. (#3287)
In a situation when we have lots of buckets the bootup time
might have slowed down a bit but during this situation the
servers quickly going up and down would be an in-transit state.

Certain calls which do not use quorum like `readXLMetaStat`
might return an error saying `errDiskNotFound` this is returned
in place of expected `errFileNotFound` which leads to an issue
where server doesn't start.

To avoid this situation we need to ignore them as safe values
to be ignored, for the most part these are network related errors.

Fixes #3275
2016-11-19 17:37:57 -08:00
Krishnan Parthasarathi
81eb7c0301 Have separate directory names for docker-compose and Dockerfile (#3292) 2016-11-19 10:42:22 -08:00
Anis Elleuch
4a926d19a7 Add GnuTLS config documentation for Windows (#3285) 2016-11-18 12:03:23 -08:00
Bala FA
05dc52a206 fix: use constants for access/secret key min/max length (#3271) 2016-11-16 17:33:55 -08:00
Harshavardhana
c91d3791f9 heal: Add healing support for bucket, bucket metadata files. (#3252)
This patch implements healing in general but it is only used
as part of quickHeal().

Fixes #3237
2016-11-16 16:42:23 -08:00
Bala FA
df8153859c docs: Add comments of using canonical import paths. (#3269) 2016-11-16 16:23:32 -08:00
Aditya Manthramurthy
2f43709f85 Prevent gorilla mux from normalizing path (Fixes #3256) (#3268)
Also fix test to not use a bucket name with a leading slash - this
causes the bucket name to become empty and go to an unintended API
call (listbuckets).
2016-11-16 16:23:22 -08:00
Bala FA
61d67a061c tests: add unit test for DeleteMultipleObjectsHandler. (#3267)
Fixes #3058
2016-11-16 09:46:09 -08:00
Harshavardhana
1b85302161 Fix spelling and golint errors. (#3266)
Fixes #3263
2016-11-15 18:14:23 -08:00
Anis Elleuch
6512d9978e Add support of user self signed certificates
Additionally add documentation about how to configure TLS with Minio
2016-11-15 16:15:23 -08:00
Aditya Manthramurthy
e216201901 Remove control command from minio binary (Fixes #3264) (#3265) 2016-11-15 13:39:02 -08:00
Krishnan Parthasarathi
7abcededf2 Add tests for storage rpc handlers (#3262) 2016-11-15 12:12:06 -08:00
Harshavardhana
398421b9f5 xl/bootup: Server bootup shouldn't return for missing buckets. (#3255)
Ref #3196
2016-11-14 15:45:00 -08:00
Anis Elleuch
b8f0d9352f signature-v2: encode path and query strings when calculating signature (#3253) 2016-11-14 10:23:21 -08:00
Harshavardhana
f234c35020 lock: slice length of lock clients should be precisely urls. (#3254)
This patch fixes a possible bug, reproduced rarely only seen
once.

```
panic: runtime error: index out of range

goroutine 136 [running]:
panic(0xac1a40, 0xc4200120b0)
    /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/minio/minio/vendor/github.com/minio/dsync.lock.func1(0xc4203d2240, 0x4, 0xc420474080, 0x4, 0x4, 0xc4202abb60, 0x0, 0xa86d01, 0xefcfc0, 0xc420417a80)
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:170 +0x69b
created by github.com/minio/minio/vendor/github.com/minio/dsync.lock
    /go/src/github.com/minio/minio/vendor/github.com/minio/dsync/drwmutex.go:191 +0xf4
```
2016-11-14 10:18:56 -08:00
koolhead17
3bcf7b7593 Docs: Modified Docker instruction adding distributed image tag. (#3250) 2016-11-13 13:40:56 -08:00
Anis Elleuch
0c042a622a Return objects content types in Web List Objects handler (#3249) 2016-11-13 12:26:40 -08:00
Harshavardhana
c57a358c9d Validate date header only for Signed{,V2} and StreamingSigned. (#3248)
Fixes #2941
2016-11-13 12:08:24 -08:00
Anis Elleuch
380d6c6435 Use getObjectInfo() in both FS and XL ListObjects() to simplify and to return complete object information (#3247) 2016-11-13 11:48:02 -08:00
Harshavardhana
716316f711 Reduce number of envs and options from command line. (#3230)
Ref #3229

After review with @abperiasamy we decided to remove all the unnecessary options

- MINIO_BROWSER (Implemented as a security feature but now deemed obsolete
  since even if blocking access to MINIO_BROWSER, s3 API port is open)
- MINIO_CACHE_EXPIRY (Defaults to 72h)
- MINIO_MAXCONN (No one used this option and we don't test this)
- MINIO_ENABLE_FSMETA (Enable FSMETA all the time)

Remove --ignore-disks option - this option was implemented when XL layer
 would initialize the backend disks and heal them automatically to disallow
 XL accidentally using the root partition itself this option was introduced.

This behavior has been changed XL no longer automatically initializes
`format.json`  a HEAL is controlled activity, so ignore-disks is not
useful anymore. This change also addresses the problems of our documentation
going forward and keeps things simple. This patch brings in reduction of
options and defaulting them to a valid known inputs.  This patch also
serves as a guideline of limiting many ways to do the same thing.
2016-11-11 16:40:55 -08:00
Anis Elleuch
98e79b4b50 Use 307 StatusTemporaryRedirect to redirect clients from http to https with forcing them to keep HTTP Verb (#3246) 2016-11-11 15:04:51 -08:00
Anis Elleuch
e2216a0936 exponentialBackoffWait returns zero after some retries, limit attempt number inside (#3245) 2016-11-11 09:49:30 -08:00
Harshavardhana
e8d9d710d0 rpc: Protect racy access of internal auth states. (#3238)
Fixes #3232
2016-11-11 00:14:32 -08:00
Harshavardhana
a8ab02a73a v4/presign: Fix presign requests when there are more signed headers. (#3222)
This fix removes a wrong logic which fails for requests which
have more signed headers in a presign request.

Fixes #3217
2016-11-10 21:57:15 -08:00
Aditya Manthramurthy
cf022de4d5 Add tests for s3PeerAPIHandlers (Fixes #3067) (#3242) 2016-11-10 16:43:04 -08:00
Bala FA
3995e21c5b fix: Ignore object not found error in RemoveObject() in web-handler. (#3228)
Fixes #3181
2016-11-10 15:02:03 -08:00
Harshavardhana
2f7fb78692 rpc: Our rpcClient should make an attempt to reconnect. (#3221)
rpcClient should attempt a reconnect if the call fails
with 'rpc.ErrShutdown' this is needed since at times when
the servers are taken down and brought back up.

The hijacked connection from net.Dial is usually closed.

So upon first attempt rpcClient might falsely indicate that
disk to be down, to avoid this state make another dial attempt
to really fail.

Fixes #3206
Fixes #3205
2016-11-10 07:44:41 -08:00
Bala FA
cf2fb30ac7 event: Add event notification for object deletion in web browser. (#3226)
Fixes #3201
2016-11-10 07:42:55 -08:00
Harshavardhana
51d1e6f75b Fix missing CompleteMultipartUpload Etag. (#3227)
Fixes #3224
2016-11-10 07:41:02 -08:00
Harshavardhana
2f373684f5 Fix the server startup messages and help text. (#3211) 2016-11-09 23:37:12 -08:00
Anis Elleuch
5741a53d46 More portable way to list files to be spellchecked and include docs/ directory (#3220) 2016-11-09 15:17:47 -08:00
Aditya Manthramurthy
dd0698d14c Improve namespace lock API: (#3203)
- abstract out instrumentation information.
- use separate lockInstance type that encapsulates the nsMutex, volume,
  path and opsID as the frontend or top-level lock object.
2016-11-09 10:58:41 -08:00
Harshavardhana
3e67bfcc88 heal: Print heal command appropriately without export path. (#3208)
Fixes #3204
2016-11-09 10:50:14 -08:00
Anis Elleuch
ea579f5b69 Avoid shutdown fs multiple times and create a new fs in each loop (#3213) 2016-11-09 10:10:14 -08:00
Anis Elleuch
daf6f3a5c0 Fix path comparing in checkgopath (#3215) 2016-11-09 10:09:42 -08:00
Aditya Manthramurthy
d44e9d6da9 Prevent weird messages from rpc lib on start (Fixes #3209): (#3212)
This is done by not making the methods of the BucketMetaState interface
as methods (via type nesting) on the type implementing
RPCs (s3PeerAPIHandlers).
2016-11-08 23:47:44 -08:00
Bala FA
9c2cfb5cb6 tests: Add missing unit test cases for AbortMultipartUploadHandler(). (#3200)
Fixes #3070
2016-11-08 16:25:00 -08:00
Bala FA
91a0ade908 tests: add unit test for HeadObjectHandler (#3197)
Fixes #3068
2016-11-07 16:02:27 -08:00
Bala FA
5eb4002bf7 cleanup build scripts. (#3192) 2016-11-07 13:28:41 -08:00
Aditya Manthramurthy
85a5c358d8 Add bucket metadata state client/handler (Fixes #3022) (#3152)
- Adds an interface to update in-memory bucket metadata state called
  BucketMetaState - this interface has functions to:
     - update bucket notification configuration,
     - bucket listener configuration,
     - bucket policy configuration, and
     - send bucket event

- This interface is implemented by `localBMS` a type for manipulating
  local node in-memory bucket metadata, and by `remoteBMS` a type for
  manipulating remote node in-memory bucket metadata.

- The remote node interface, makes an RPC call, but the local node
  interface does not - it updates in-memory bucket state directly.

- Rename mkPeersFromEndpoints to makeS3Peers and refactored it.

- Use arrayslice instead of map in s3Peers struct

- `s3Peers.SendUpdate` now receives an arrayslice of peer indexes to
  send the request to, with a special nil value slice indicating that
  all peers should be sent the update.

- `s3Peers.SendUpdate` now returns an arrayslice of errors, representing
  errors from peers when sending an update. The array positions
  correspond to peer array s3Peers.peers

Improve globalS3Peers:

- Make isDistXL a global `globalIsDistXL` and remove from s3Peers

- Make globalS3Peers an array of (address, bucket-meta-state) pairs.

- Fix code and tests.
2016-11-07 12:09:24 -08:00
Harshavardhana
33c771bb3e tests: Add tests for browser peer rpc. Fixes #3062 (#3189) 2016-11-07 11:43:35 -08:00
Karthic Rao
286a8924fd Add leak detection to object-handler tests. (#3195) 2016-11-06 21:53:50 -08:00
Karthic Rao
efca29b00e Fix typos and comments in leak_detect_test.go. (#3193) 2016-11-06 21:02:29 -08:00
Harshavardhana
9161016962 tests: Improve coverage on signature v4 tests. (#3188)
Fixes #3065
2016-11-06 11:47:16 -08:00
Anis Elleuch
5ff30777e1 Rewrite connection muxer peek process to avoid server blocking by silent clients (#3187) 2016-11-06 11:41:01 -08:00
Anis Elleuch
754c0770d6 Merge ListenAndServe and ListenAndServeTLS for simplification purpose (#3186) 2016-11-05 20:32:13 -07:00
Harshavardhana
1105508453 connection muxer should use bufio.Reader to be simpler. (#3177) 2016-11-05 12:57:31 -07:00
Harshavardhana
1ba497950c Fix net.Listener to fully close the underlying socket. (#3171)
Leads to races and accepting connections. This patch implements
a way to reject accepting new connections.
2016-11-05 09:43:28 -07:00
Harshavardhana
bf3c93a8cc Fix docker file to use binary endpoint. (#3180) 2016-11-04 18:25:30 -07:00
Aditya Manthramurthy
eb1bc67db1 Refactor: simplify signV4TrimAll() (#3179) 2016-11-04 13:52:22 -07:00
Krishna Srinivas
8408dfaa6c bootup-validation: Allow port configuration only using --address option. (#3166) 2016-11-04 12:14:19 -07:00
Harshavardhana
d192044915 router: PathPrefix router was wrong. (#3172) 2016-11-04 12:13:22 -07:00
Anis Elleuch
d9bab6b3bd sigv4: Trim and shrink spaces in headers values (#3162) 2016-11-03 16:41:25 -07:00
koolhead17
4503491a0a docs:Browser/added storageInfo content to the doc. (#3146) 2016-11-03 01:58:55 -07:00
Anis Elleuch
e6965ca066 Quit initializing disks process when term signal is invoked (#3163) 2016-11-02 15:27:36 -07:00
Harshavardhana
d9674f7524 Improve coverage of web-handlers.go (#3157)
This patch additionally relaxes the requirement for
accesskeys to be in a regexy set of values.

Fixes #3063
2016-11-02 14:45:11 -07:00
koolhead17
f024deb1f8 docs: Fixed path for docker compose (#3156) 2016-11-02 08:54:20 -07:00
Anis Elleuch
79601d27f2 Use endpoint url when printing disks status in distributed mode (#3151) 2016-11-02 08:51:06 -07:00
Aditya Manthramurthy
226a69fe15 Add test for updateUploadJSON (Fixes #3060) (#3155) 2016-11-01 20:04:32 -07:00
Harshavardhana
9bb799462e Start all listeners when a given host resolves to multiple IPs. (#3145)
Default golang net.Listen only listens on the first IP when
host resolves to multiple IPs.

This change addresses a problem for example your ``/etc/hosts``
has entries as following

```
127.0.1.1  minio1

192.168.1.10 minio1
```

Trying to start minio as

```
minio server --address "minio1:9001" ~/Photos
```

Causes the minio server to be bound only to "127.0.1.1" which
is an incorrect behavior since we are generally interested in
`192.168.1.10` as well.

This patch addresses this issue if the hostname is resolvable
and gives back list of addresses associated with that hostname
we just bind on all of them as it is the expected behavior.
2016-11-01 15:38:28 -07:00
Karthic Rao
8bffa78f7f Fix Instance type during benchmarks. (#3147)
- The benchmark initialization function was not taking into account the
  instance type (FS/XL), was using XL ObjectLayer even for FS
  benchmarks.
- This was leading to incorrect benchmark results for FS related
  benchmarks.
- The fix takes into account the instance type (FS/XL) and correctly
  returns FS backend for FS benchmarks.
2016-11-01 10:21:16 -07:00
Anis Elleuch
4b302173ae fs: Remove object append file if found since it is not always used (#3139) 2016-11-01 00:56:03 -07:00
Karthic Rao
9db2205db3 Fix List-Object Benchmark failure. (#3142) 2016-10-31 13:57:30 -07:00
Aditya Manthramurthy
6914fe436c Remove unused function. (#3143) 2016-10-31 12:18:15 -07:00
Anis Elleuch
807cc3c28d Tracing locking errors for better debugging (#3140) 2016-10-31 10:17:14 -07:00
Harshavardhana
f3c6c55719 posix: Fix windows performance issues. (#3132)
Do not attempt to fetch volume/drive information for
each i/o situation. In our case we do this in all calls
`posix.go` this in-turn created a terrible situation for
windows. This issue does not affect the i/o path on Unix
platforms since statvfs calls are in the range of micro
seconds on these platforms.

This verification is only needed during startup and we
let things fail at a later stage on windows.
2016-10-31 09:34:44 -07:00
Harshavardhana
a773a6dce6 docker: Fix docker build. 2016-10-31 02:51:00 -07:00
Harshavardhana
62dcee3b14 dist: Moved systemd scripts to minio-systemd. (#3136)
Removing this from the repo.
2016-10-31 02:37:32 -07:00
Harshavardhana
490159ea89 docker: Move the docker document and compose to a separate dir. (#3135) 2016-10-31 02:08:16 -07:00
Karthic Rao
3e8cb8c937 build: ineffassign fixes. (#3134) 2016-10-30 23:32:46 -07:00
Aditya Manthramurthy
dd6ecf1193 Read/write quorum algo with uploads.json in xl (Fixes #3123) (#3124)
- Reads and writes of uploads.json in XL now uses quorum for
  newMultipart, completeMultipart and abortMultipart operations.

- Each disk's `uploads.json` file is read and updated independently for
  adding or removing an upload id from the file. Quorum is used to
  decide if the high-level operation actually succeeded.

- Refactor FS code to simplify the flow, and fix a bug while reading
  uploads.json.
2016-10-30 09:27:29 -07:00
Anis Elleuch
90e1803798 Fix docker-compose to follow the new server args style (#3128) 2016-10-29 16:53:09 -07:00
Anis Elleuch
a47ce7ab22 Add support of fallocate for FS and XL backends (#3032) 2016-10-29 12:44:44 -07:00
Krishna Srinivas
0b3282ac9f Logging: errorIf fatalIf print in the format [file.go:82:funcName()] (#3127) 2016-10-29 02:34:16 -07:00
Krishna Srinivas
79b98b5c25 boot: getPath() should take care of simple directory exports. (#3122)
fixes #3120
2016-10-28 13:08:56 -07:00
Krishnan Parthasarathi
6a57f2c1f0 XL: Add more information to panic msg (#3119) 2016-10-28 08:46:03 -07:00
Krishna Srinivas
0fa2477cb0 boot: check parameter syntax before initializing the system. (#3114)
fixes #3112
2016-10-28 08:45:32 -07:00
Anis Elleuch
7ffb337cd7 Update control cmd USAGE to indicate that it is possible to put access secret keys in URL (#3115) 2016-10-27 13:26:11 -07:00
Karthic Rao
a9dd2793f9 Better formatting and adding systemd config to run Minio server on custom port. (#3113) 2016-10-27 10:04:06 -07:00
Harshavardhana
9e2d0ac50b Move to URL based syntax formatting. (#3092)
For command line arguments we are currently following

- <node-1>:/path ... <node-n>:/path

This patch changes this to

- http://<node-1>/path ... http://<node-n>/path
2016-10-27 03:30:52 -07:00
Aditya Manthramurthy
30dc11a931 No listener.json for single-node mode (Fixes #3052) (#3108)
In FS or single-node XL mode, there is no need to save listener
configuration to persistent storage. As there is only one server, if it
is restarted, any connected listenBucketAPI clients were disconnected
and will have to reconnect - so there is nothing to actually store.

This incidentally solves #3052 by avoiding the problem.
2016-10-26 20:13:00 -07:00
Anis Elleuch
a15dc5fed5 Print message when creating the config file (#3089) 2016-10-26 18:44:22 -07:00
Anis Elleuch
f7c20b97a1 control cmds: Extract access and secret keys from URL if specified (#3109) 2016-10-26 18:42:12 -07:00
Harshavardhana
e9c45102b0 posix: Use sync.Pool buffers to copy in large buffers. (#3106)
These fixes are borrowed from the fixes required for GlusterFS i/o throughput.
2016-10-26 17:14:05 -07:00
Anis Elleuch
8871eb8e1e Show offline nodes after a fixed number of init retry (#3107) 2016-10-26 16:09:06 -07:00
Krishna Srinivas
0f32efb825 PostPolicy - rename of files/functions + add testcases (#3104) 2016-10-26 10:15:57 -07:00
Karthic Rao
63f1b4fdf4 mispell fixes. (#3100) 2016-10-26 08:46:14 -07:00
Krishnan Parthasarathi
31f2db6880 Remove leftover debug statement from PutObject StreamingSignature unit-test (#3099) 2016-10-26 03:17:47 -07:00
Krishnan Parthasarathi
2c9b406f6c Add TLS based tests to functional test suite (#3083) 2016-10-26 02:30:31 -07:00
Krishnan Parthasarathi
12cd2da265 Add PutObjectHandler unit tests covering failure cases (#3096) 2016-10-26 02:06:22 -07:00
Harshavardhana
485c0ea8bf tests: Combine v2 tests with the Suite itself. (#3088) 2016-10-25 13:34:14 -07:00
Krishnan Parthasarathi
49ba07d1d6 Use net.ParseCIDR instead of custom-built parsers (#3055)
Removes avoidable conversion to and from net.IP to string.
2016-10-25 11:14:47 -07:00
Krishna Srinivas
35e541e0b1 content-length-range policy should be honored for the uploaded object sizes. (#3076) 2016-10-24 23:47:03 -07:00
Aditya Manthramurthy
3977d6b7bd Lock bucket while modifying its metadata (Fixes #2979) (#3019)
- When modifying notification configuration
- When modifying listener configuration
- When modifying policy configuration

With this change we also stop early checking if the bucket exists, since
that uses a Read-lock and causes a deadlock due to the outer Write-lock.
2016-10-24 19:52:24 -07:00
Harshavardhana
0905398459 Fix benchmark tests. (#3082)
Fixes #3081
2016-10-24 18:45:06 -07:00
Aditya Manthramurthy
f41faf96b7 Fix newMultipartUpload to not leave stale uploads.json (Fixes #3014) (#3079) 2016-10-24 17:37:18 -07:00
Bala FA
36639b65a9 rename completeMultipartMD5() into getCompleteMultipartMD5(). (#3051) 2016-10-24 13:56:13 -07:00
Harshavardhana
7fc598b73f Fix user-agent prefix to have docker instead of suffix. (#3074) 2016-10-24 13:44:15 -07:00
Krishna Srinivas
21d41ad7fd init[windows]: Fix to handle the case when export path is a relative path. (#3054)
ex. to handle "minio server export"
2016-10-24 08:26:28 -07:00
Harshavardhana
fe56220d1a Do not print nil when hostname is provided as --address (#3053)
Fixes #3018
2016-10-23 23:55:12 -07:00
Harshavardhana
5782ec3ada Fix peers and web UIVersion validation. (#3048) 2016-10-23 12:32:35 -07:00
Krishnan Parthasarathi
8839c5105a Pass values to closures esp. when passed to defer statement. (#3050)
opsID, a variable on the stack, changes over the course of
Completemultipartupload function in xl-v1-multipart.go.  This was
being used in a function closure which was passed to defer
statement. The variables used in the closure depend on their values at
the time of evaluation which is indeterminate behaviour. It is
incorrect to depend on values of variables on stack at the end of
function, when deferred functions are executed.
2016-10-23 09:57:52 -07:00
Harshavardhana
e293f079f5 github: Add PR and issue templates. (#3046) 2016-10-22 17:35:52 -07:00
Anis Elleuch
6c2d5e3d05 Correct the number of failed disks that we can withstand in startup message (#3045) 2016-10-22 10:36:50 -07:00
Krishna Srinivas
5999a23d3e When object whose size is greater than 5G is uploaded using presigned POST we should return error. (#3033)
fixes #2961
2016-10-22 09:05:01 -07:00
Krishna Srinivas
e51be73ac7 PresignedPost: Support for Signature V2 presigned POST Policy. (#3043)
fixes #2993
2016-10-22 08:57:12 -07:00
Harshavardhana
4b5b363c6c tests: Fix race between SetBucketListenerConfig and eventNotifyForBucketNotifications (#3041) 2016-10-22 02:35:33 -07:00
Krishna Srinivas
f2b0c08e34 logging: print file:line:funcName of the caller of errorIf and fatalIf (#3035) 2016-10-22 02:24:34 -07:00
Harshavardhana
83b364891d tests: Fix a potential race in ListenBucketNotificationHandler. (#3040) 2016-10-21 22:56:27 -07:00
Karthic Rao
87af2dbc43 dist: Adding systemd script for running Minio distributed. (#3034) 2016-10-21 20:56:56 -07:00
Justin Clift
885bac330b Added a "catch all" documentation link (#3038)
This should help guide other newbies ;)
2016-10-21 19:31:21 -07:00
Harshavardhana
e3ab478c70 tests: Fix a potential race in RemoveBucketNotification. (#3037)
Fixes #3036
2016-10-21 17:12:56 -07:00
Harshavardhana
ecaccefd2e tests: Implement GetBucketNotification handler tests. (#3029) 2016-10-21 02:39:37 -07:00
Harshavardhana
ece559afe2 api: Do not use sqs for ListenBucketNotification. (#3023)
Add more tests. Fixes #3024
2016-10-21 01:25:17 -07:00
Krishna Srinivas
d3aaf50a40 posix: Split on ":" in path d:\export makes minio use wrong disk. (#3027)
As the host/path split happens at a higher layer now, split at posix is not needed.
fixes part of #2987
2016-10-20 23:39:33 -07:00
Karthic Rao
43ce028840 Point link of the docs to docs.minio.io. (#3025) 2016-10-20 22:28:22 -07:00
Krishna Srinivas
32c3a558e9 distributed-XL: Support to run one minio process per export even on the same machine. (#2999)
fixes #2983
2016-10-20 18:31:02 -07:00
Anis Elleuch
41f9ab1c69 Translate storage access denied error to S3 Access Denied response (#3015) 2016-10-20 16:09:55 -07:00
Aditya Manthramurthy
8876e0a80a Delete bucket listener config file from disk (#3016) 2016-10-20 16:09:19 -07:00
Anis Elleuch
c21ac80268 Validate access/secret keys found in the config file and enhance invalid keys messages (#3017) 2016-10-20 16:07:24 -07:00
Frank
0e2cd1a64d Added clear subcommand for control lock (#3013)
Added clear subcommand for control lock with following options:

```
  3. Clear lock named 'bucket/object' (exact match).
    $ minio control lock clear http://localhost:9000/bucket/object

  4. Clear all locks with names that start with 'bucket/prefix' (wildcard match).
    $ minio control lock --recursive clear http://localhost:9000/bucket/prefix

  5. Clear all locks older than 10minutes.
    $ minio control lock --older-than=10m clear http://localhost:9000/

  6. Clear all locks with names that start with 'bucket/a' and that are older than 1hour.
    $ minio control lock --recursive --older-than=1h clear http://localhost:9000/bucket/a
```
2016-10-20 13:15:28 -07:00
Aditya Manthramurthy
6274727b71 Pick up server address from --address option (#3002) (#3008)
This makes sure that when SSL is enabled (for FS/single node mode),
the server address is picked up from the --address option (that needs
to include the hostname for SSL verification, and has to be input
appropriately by user), instead of just using ":<port>".
2016-10-20 11:39:10 -07:00
Harshavardhana
95567c68bf posix: Do not print errors in expected errors. (#3012)
Fixes #3011
2016-10-20 09:26:18 -07:00
Anis Elleuch
c189337b6e rpc: Support SNI in TLS certificates (#3009) 2016-10-20 07:43:31 -07:00
Krishnan Parthasarathi
6fc81dc162 Delete temp object/part when PutObject{,Part} fails (#3004) 2016-10-19 22:52:03 -07:00
Krishnan Parthasarathi
7d50361ca9 Move housekeeping before object layer initialization (#3001)
In a distributed setup that the server should not perform any operation
on the storage layer after it is exported via RPC. e.g, cleaning up of
temporary directories under .minio.sys/tmp may interfere with ongoing
PUT objects being served by the distributed setup.
2016-10-19 19:59:48 -07:00
Frank
19c51f3f3c Added ForceUnlock to namespace-lock (#2990) 2016-10-19 09:27:36 -07:00
Aditya Manthramurthy
c3bbadacbf Improve Peer RPC error handling (Fixes #2992) (#2995)
* Check for RPC connection shutdown and try again just once.

* Refactor SendRPC to use sync.WaitGroup
2016-10-18 21:26:58 -07:00
Anis Elleuch
2208992e6a More informative message when erasure fails to read a part of an object (#2989) 2016-10-18 13:09:26 -07:00
Anis Elleuch
bbba8e432a Add ssl support to s3/web peers connections (#2988) 2016-10-18 11:46:33 -07:00
Harshavardhana
39331b6b4e xl: GetCheckSumInfo() shouldn't fail if hash not available. (#2984)
In a multipart upload scenario disks going down and coming backup
can lead to certain parts missing on the disk/server which was
going down. This is a valid case since these blocks can be
missing and should be healed through heal operation. But we are
not supposed to fail prematurely since we have enough data on
the other disks as well within read-quorum.

This fix relaxes previous assumption, fixes a major corruption
issue reproduced by @vadmeste.

Fixes #2976
2016-10-18 11:13:25 -07:00
Mike Ralphson
6e748cb1cf Report when invalid bucket names are skipped in FS backend. (#2947) 2016-10-18 01:42:46 -07:00
Anis Elleuch
2005d656e6 Properly load creds from env and save them when server cmd is executed (#2970) 2016-10-17 23:14:41 -07:00
Aditya Manthramurthy
0f26ec8095 Propagate creds change to cluster (Fixes #2855) (#2929) 2016-10-17 20:18:08 -07:00
Harshavardhana
8d2347bc7b storage: DeleteFile should return errFileNotFound for ENOENT. (#2978) 2016-10-17 16:38:46 -07:00
Aditya Manthramurthy
0ff359ca0e Fix early init. problem for notifications (Fixes #2972) (#2977) 2016-10-17 16:38:29 -07:00
Harshavardhana
f8e13fb00e server: Startup sequence should be more idempotent. (#2974)
Fixes #2971 - honors ignore-disks option properly.
Fixes #2969 - change the net.Dial to have a timeout of 3secs.
2016-10-17 14:31:33 -07:00
Harshavardhana
686a610fc3 api: Nanosecond precision for API responses is valid with S3. (#2957)
Wqe need to be compatible as well fixes #2955
2016-10-17 08:44:55 -07:00
Mike Ralphson
7fc1685b7a Allow Travis builds from GitHub forks (#2958)
Set the go_import_path explicitly. See
https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-10-17 08:42:20 -07:00
Krishnan Parthasarathi
b89609dc2e XL: Filter out md5Sum from user defined headers (#2962) 2016-10-17 08:41:33 -07:00
Anis Elleuch
fa50312220 Avoid returning disk corrupted by servers in the middle of init all disks formats (#2964) 2016-10-17 08:39:55 -07:00
Harshavardhana
fee3f99a6e xl: heal bucket should validate if bucket exists first. (#2953)
Fixes #2944
2016-10-17 02:10:23 -07:00
Frank
ea406754a6 New dsync and added ForceUnlock to lock rpc server (#2956)
* Update dsync and added ForceUnlock function
* Added test cases for ForceUnlock
2016-10-17 01:53:29 -07:00
Aditya Manthramurthy
d02cb963d5 Fix listen-bucket (Fixes #2942) (#2949)
Don't close socket while re-initializing notify-listeners, as the rpc
client object is shared between notify-listeners and peer clients.

Also, improves SendRPC() readability by using GetPeerClient().
2016-10-16 20:52:10 -07:00
Anis Elleuch
334cdb5d64 XL total/free space calculation is done inside xl module (#2945) 2016-10-16 14:24:15 -07:00
Harshavardhana
a681af6953 Update minio browser with new changes. (#2940)
- Bucket policy set/unset support.
- Shareable URL support.
- Delete object support.
2016-10-15 08:51:53 -07:00
Anis Elleuch
5c3639c1b7 Redirect /minio to /minio/ when requests come from browsers (#2937) 2016-10-15 06:21:51 -07:00
Krishna Srinivas
903574db90 copy-object: Do not use ETag of source as MD5 as it will not be MD5 if source was uploaded as multipart. (#2938)
fixes #2934
2016-10-15 06:20:55 -07:00
Anis Elleuch
f463d3ce42 Fix a crash when service shutdown is signaled and object API is not ready yet (#2939) 2016-10-15 06:20:16 -07:00
Aditya Manthramurthy
17eeec6895 Bucket policy propagation (Fixes #2930) (#2932)
Fixes a serialisation bug - encoding/gob does not directly support
serializing `map[string]interface{}`, so we serialise to JSON and send a
byte array in the RPC call, and deserialize and update on the receiver.
2016-10-14 22:49:51 -07:00
Karthic Rao
070d3610ff tests: V2 Signature tests for object-handlers. (#2931) 2016-10-14 20:52:46 -07:00
Harshavardhana
f22862aa28 heal: Refactor heal command. (#2901)
- return errors for heal operation through rpc replies.
  - implement rotating wheel for healing status.

Fixes #2491
2016-10-14 19:57:40 -07:00
koolhead17
18be3bc95a docs: added space in README.md so that its compatible with doctor. (#2927) 2016-10-14 13:20:33 -07:00
Harshavardhana
2f520ed92f Remove errors package, add comments and simplify. (#2925) 2016-10-14 12:31:00 -07:00
Mateusz Gajewski
c03ce0f74a Display SSL expiry warnings (#2925) 2016-10-14 12:30:36 -07:00
Krishna Srinivas
0320a77dc0 HealBucket: create the bucket if it is missing in one of the disks. (#2924) 2016-10-14 11:12:17 -07:00
koolhead17
3349153058 docs: added space in source download steps so it appears as desired in (#2923)
doctor.
2016-10-14 08:49:12 -07:00
Harshavardhana
5e86352464 doc: Fix docker.md instructions and words. 2016-10-13 21:34:03 -07:00
Harshavardhana
18d125ef1c doc: Redo install instructions (#2922) 2016-10-13 19:44:27 -07:00
Aditya Manthramurthy
31be826f51 Fix missing error check for jsonrpc.Server.RegisterService() (#2921) 2016-10-13 17:34:10 -07:00
Harshavardhana
eb372d53df Fix docker release titles 2016-10-13 16:24:18 -07:00
Harshavardhana
1788c58d5c Add docker edge instructions 2016-10-13 16:19:14 -07:00
Karthic Rao
17e49a9ed2 signature-v2 fix. (#2918)
- Return errors similar to V4 Sign processsing.
- Return ErrMissing fields when Auth Header fields are missing.
- Return InvalidAccessID when accessID doesn't match.

* tests: Adding V2 signature tests for bucket handler API's.
2016-10-13 09:25:56 -07:00
Aditya Manthramurthy
0aabc1d8d9 Use Peer RPC to propagate bucket policy changes (#2891) 2016-10-13 09:19:04 -07:00
Harshavardhana
55f6828750 Do not print update message unless there is an update. (#2919) 2016-10-13 09:17:08 -07:00
Aditya Manthramurthy
6303f26330 Protect map from concurrent access (Fixes #2915) (#2916)
Protects the Peers RPC clients map from concurrent access to fix a data race condition.
2016-10-13 01:33:50 -07:00
Krishnan Parthasarathi
b59bac670a Handle err returned by rpc.Server.RegisterName (#2910) 2016-10-12 23:13:24 -07:00
Anis Elleuch
84acc820c7 Fix free drive space calculation in XL mode (#2917) 2016-10-12 20:22:15 -07:00
Harshavardhana
92858c7db2 Fix docker documentation. 2016-10-12 18:31:23 -07:00
Harshavardhana
fdaa129a5b Fix dockerfile container image. (#2892) 2016-10-12 18:09:08 -07:00
Anis Elleuch
df59967f59 Avoid checking date header of web requests by properly applying generic handlers (#2914) 2016-10-12 12:58:36 -07:00
Mateusz Gajewski
73982c8cb6 Listen bucket notification for multiple prefixes/suffixes (#2911)
* Listen bucket notification for multiple prefixes/suffixes

* After review fixes
2016-10-12 11:02:15 -07:00
Aditya Manthramurthy
6199aa0707 Peer RPCs for bucket notifications (#2877)
* Implements a Peer RPC router that sends info to all Minio servers in the cluster.
* Bucket notifications are propagated to all nodes via this RPC router.
* Bucket listener configuration is persisted to separate object layer
  file (`listener.json`) and peer RPCs are used to communicate changes
  throughout the cluster.
* When events are generated, RPC calls to send them to other servers
  where bucket listeners may be connected is implemented.
* Some bucket notification tests are now disabled as they cannot work in
  the new design.
* Minor fix in `funcFromPC` to use `path.Join`
2016-10-12 01:03:50 -07:00
Krishnan Parthasarathi
a5921b5743 Use same timestamp for all chunks in chunked signature (#2908) 2016-10-11 23:46:51 -07:00
Karthic Rao
f0538dbb5c fix broken link for Go Installation in CONTRIBUTING.md (#2907) 2016-10-11 22:19:35 -07:00
Karthic Rao
ff91ecb177 tests: Adding unknown signature type test for API handlers. (#2905) 2016-10-11 20:38:10 -07:00
Frank
a6357502c1 Correct typo in error string (#2902) 2016-10-11 08:56:02 -07:00
Harshavardhana
fa8ea41cd9 lock/instrumentation: Cleanup and print in user friendly form. (#2807) 2016-10-11 00:50:27 -07:00
Karthic Rao
3ac6790ca2 tests: Add Object Layer nil test for bucket-handler API's (#2899) 2016-10-11 00:00:02 -07:00
Krishna Srinivas
268b96058f ns-lock: lock namespace during FS object operations. (#2896) 2016-10-10 10:20:04 -07:00
Frank
0d031c432b Fix typo in serverVersion (#2894) 2016-10-10 10:11:56 -07:00
Frank
6e8f3224c5 Test coverage for lock rpc server (#2893)
* Add test coverage for removeEntry and removeEntryIfExists
* Initial test framework for Lock/Unlock functionality
* Add clarification comments
* Add test coverage code for RLock() and RUnlock()
* Add test coverage for Expired() function
* Have all lock-rpc-server test functions start with the same prefix
* Properly initialize JWT security token
2016-10-10 10:11:29 -07:00
Karthic Rao
9c53e9f4c3 tests: Enhance coverage for bucket policy handlers. (#2895) 2016-10-10 09:29:56 -07:00
Krishnan Parthasarathi
2d5e988a6d Refactor streaming signatureV4 w/ state machine (#2862)
* Refactor streaming signatureV4 w/ state machine

- Used state machine to make transitions between reading chunk header,
  chunk data and trailer explicit.

* debug: add print/panic statements to gather more info on CI failure

* Persist lastChunk status between Read() on ChunkReader

... remove panic() which was added as interim aid for debugging.

* Add unit-tests to cover v4 streaming signature
2016-10-10 01:42:32 -07:00
Harshavardhana
3cfb23750a control: Implement service command 'stop,restart,status'. (#2883)
- stop - stops all the servers.
- restart - restart all the servers.
- status - prints status of storage info about the cluster.
2016-10-09 23:03:10 -07:00
Anis Elleuch
57f75b1d9b Ignore copy conditions when ETag is not available (#2888) 2016-10-09 16:21:42 -07:00
Anand Babu (AB) Periasamy
4560cbc20c List docker as first example 2016-10-09 13:06:25 -07:00
Karthic Rao
e213172431 tests: Missing anonymous tests for bucket-handlers. (#2885) 2016-10-09 09:21:37 -07:00
Krishna Srinivas
f5f007e183 Test: Add test case for xl.HealObject() (#2884)
fixes #2842
2016-10-08 17:08:17 -07:00
Karthic Rao
09463265ce tests: Adding anonymous requests tests for bucket policy handlers. (#2882) 2016-10-08 01:04:26 -07:00
Karthic Rao
8f4cf2a7d0 tests: anonymous/unsigned tests for object handler API's . (#2881) 2016-10-07 23:28:50 -07:00
Karthic Rao
30183c4a9a tests: cleanup and unsigned request test. (#2880)
- Cleaning up of ListMultipartUpload API test for improving readability,
  code maintainance and extensibility.
- Moving ListMultipartUploads to Go 1.7 sub tests.
- Using the new Anonymous request helper function for
  ListMultipartUploads.
2016-10-07 20:16:57 -07:00
Karthic Rao
d1df5e0ae1 tests: Add helper function for API handler anonymous request tests. (#2876)
- Add helper function for API handler anonymous request tests.
- Add PutObject Part Anonymous request case using the new helper
  function to validate its functionality.
2016-10-07 11:16:11 -07:00
Harshavardhana
f1bc9343a1 prep: Initialization should wait instead of exit the servers. (#2872)
- Servers do not exit for invalid credentials instead they print and wait.
- Servers do not exit for version mismatch instead they print and wait.
- Servers do not exit for time differences between nodes they print and wait.
2016-10-07 11:15:55 -07:00
Frank
e53a9f6cab Update vendorized version of dsync that relaxes read quorum to N/2 (#2874) 2016-10-07 08:02:59 -07:00
Karthic Rao
97f4989945 tests: cleaning up. (#2875)
- Clean up PutObjectPart and ListObjectPart API handler tests.
- Add more comments, make the tests more readable.
- Add verification for HTTP response status code.
- Initialize the test using object Layer.
- Move to Go 1.7 sub tests.
2016-10-07 08:02:37 -07:00
Harshavardhana
ed676667d0 vendor: update reedsolomon package with new changes. (#2870)
- Cached inverse matrices for better reconstruct performance.
- New error reconstruction required is returned, helpful in
  initiating healing.
2016-10-06 21:57:42 -07:00
Harshavardhana
1e5e213d24 auth: Make sure we initialize or change config before RPC requests. (#2867) 2016-10-06 13:35:56 -07:00
Karthic Rao
a8105ec068 - Test utility function for easy asserting of cases wherein objectLayer (#2865)
is `nil` in API handlers.
- Remove the existing tests for the `nil` check and use the new method
  to test for object layer being `nil`.
2016-10-06 13:34:33 -07:00
Krishna Srinivas
c6d2967b84 Doc: Document list of supported environmental varaibles. (#2864)
fixes #2773
2016-10-06 09:30:08 -07:00
Krishna Srinivas
bb9be02228 minio-browser: do not redirect to /minio if MINIO_BROWSER=off (#2863)
fixes #2837
2016-10-06 08:30:32 -07:00
Harshavardhana
64f37bbf5b rpc: Add RPC client tests. (#2858) 2016-10-06 02:30:54 -07:00
Karthic Rao
0fc96fa25c Refactor bucket policy handler test to use API test initializer. (#2859) 2016-10-06 02:02:42 -07:00
Karthic Rao
2d8c6f8288 unit test for bucketPolicyConditionMatch function. (#2857) 2016-10-06 00:23:46 -07:00
Harshavardhana
b94211bd66 api: ListObjectsV1 compliance with AWS S3. (#2856)
XSD - xml schema definition for SOAP operations
on S3 provides positional restrictions on XML
output.

Fix the response by re-arranging the positions in
accordance with S3 behavior.

Fixes #2849
2016-10-05 20:12:47 -07:00
Harshavardhana
6494b77d41 server: Add more elaborate startup messages. (#2731)
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.

This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Bala FA
63a7ca1af0 web: fix jwt token expiry set to one day by default. (#2819)
Fixes #2818
2016-10-05 10:18:55 -07:00
Krishna Srinivas
95f544657a Signature-V2: use raw resource/query from the request for signature calculation. (#2833) 2016-10-05 09:18:53 -07:00
Harshavardhana
740a919e25 config: Use migrateV8 to v9 function properly. (#2852) 2016-10-05 02:28:04 -07:00
Krishnan Parthasarathi
402c92beda Add listObjectParts test w/ unknown request signature type (#2847) 2016-10-04 07:57:35 -07:00
Karthic Rao
6a9013b97c misspell fixes. (#2835) 2016-10-04 00:09:21 -07:00
Krishnan Parthasarathi
73b50aea2d Add preSign auth type tests for ListObjectPartsHandler and PutObjectPartHandler (#2834) 2016-10-03 22:05:33 -07:00
Aditya Manthramurthy
315e66858c Add PostgreSQL notifier (#2739) (#2824)
* The user is required to specify a table name and database connection
  information in the configuration file.

* INSERTs and DELETEs are done via prepared statements for speed.

* Assumes a table structure, and requires PostgreSQL 9.5 or above due to
  the use of UPSERT.

* Creates the table if it does not exist with the given table name using
  a query like:

    CREATE TABLE myminio (
        key varchar PRIMARY KEY,
        value JSONB
    );

* Vendors some required libraries.
2016-10-03 17:29:55 -07:00
Krishnan Parthasarathi
4f902d42b2 Add unit-tests for ListObjectParts API handler (#2826)
* Add missing uploadID test
... make variables in test code unexported.
* Add ServerNotInitialized test for ListObjectPartsHandler
* Add tests for ListObjectParts with signatureV2 and Anonymous requests
* Add failure test cases for ListObjectParts
2016-10-03 08:54:57 -07:00
Krishna Srinivas
61a18ed48f sha256: Verify sha256 along with md5sum, signature is verified on the request early. (#2813) 2016-10-02 15:51:49 -07:00
Anis Elleuch
b5a6dd1395 Avoid path-cleaning policy resources for a better compliance with S3 (#2823) 2016-10-01 21:30:25 -07:00
Krishnan Parthasarathi
83e6e1060e Layer LimitReader responsibly allowing sign verification to work (#2821) 2016-10-01 09:37:40 -07:00
Krishnan Parthasarathi
ddeb8242d8 PutObjectPartHandler unit-tests (#2810) 2016-10-01 08:23:26 -07:00
Harshavardhana
a08052f640 Add docker pulls badge. 2016-09-30 19:19:19 -07:00
Harshavardhana
5ecba587f7 api: Relax object name validation. (#2814)
Fixes #2812
2016-09-30 16:56:36 -07:00
Harshavardhana
db3da97a50 signature/v2: Fix presigned requests. 2016-09-30 15:22:00 -07:00
Harshavardhana
5885ffc8ae signature: Add legacy signature v2 support transparently. (#2811)
Add new tests as well.
2016-09-30 14:32:13 -07:00
Anis Elleuch
9fb1c89f81 Add TLS encryption capability to RPC clients (#2789) 2016-09-29 23:42:37 -07:00
Anis Elleuch
1e6afac3bd Add NATS notifier (#2795) 2016-09-29 23:42:10 -07:00
Harshavardhana
64083b9227 signature: Region changes should be handled just like AWS. (#2805)
- PutBucket happens with 'us-east-1'.
- ListBuckets happens with any region.
- GetBucketLocation happens with 'us-east-1' and location is returned.
2016-09-29 15:51:00 -07:00
Krishnan Parthasarathi
5fdd768903 Make addition of TopicConfig to globalEventNotifier go-routine safe (#2806) 2016-09-28 22:46:19 -07:00
Harshavardhana
f72163f856 build: Deprecate requirement of GOROOT (#2803) 2016-09-28 18:49:16 -07:00
Krishnan Parthasarathi
428629f577 Add unit tests for server-main.go (#2802) 2016-09-28 11:19:07 -07:00
Harshavardhana
1edd74dda2 update: Deprecate the usage of update=yes query param. (#2801)
Fixes #2799
2016-09-28 02:41:21 -07:00
Krishnan Parthasarathi
740ecf530c Add PutBucketNotification, ListenBucketNotification handler unit-tests. (#2787) 2016-09-28 01:08:03 -07:00
Aditya Manthramurthy
10d2ef5449 Remove comments relating to deprecated MINIO_DEBUG envvar (#2797) 2016-09-27 18:28:46 -07:00
Aditya Manthramurthy
8ea571c7f7 Remove MINIO_DEBUG environment variable (#2794)
Removes the unimplemented settings of MINIO_DEBUG=mem and makes
MINIO_DEBUG=lock the default behaviour.
2016-09-27 14:35:43 -07:00
Harshavardhana
ca3022d545 api: Change ListenBucketNotification with new API format. (#2791)
Take prefix, suffix and events as query params.
2016-09-27 13:17:43 -07:00
Anis Elleuch
9417614a8e Recalculate free minimum disk space (#2788)
* Fix calculating free space disk by using blocks available for unprivileged user

* Use fixed minimal free disk space instead of percentage
2016-09-27 12:46:38 -07:00
Aditya Manthramurthy
70d52bbc4c Add unit test for rate-limit-handler (#2661) (#2784) 2016-09-26 21:31:12 -07:00
Harshavardhana
6aa2fc95c0 Revert "bucket: refactor policies and fix bugs related to enforcing policies. (#2766)"
This reverts commit ca5ca8332b.
2016-09-26 19:32:33 -07:00
Harshavardhana
cfbab22237 web: Remove bucket policy when we have no more statements. (#2779) 2016-09-26 03:11:22 -07:00
Harshavardhana
be0e06c0aa web: Simplify and rename GetAllBucketPolicy --> ListAllBucketPolicies. (#2778) 2016-09-25 21:53:19 -07:00
Harshavardhana
1c941fd787 rpc: Should validate server versions. (#2775)
Fixes #2764
2016-09-24 03:34:45 -07:00
Krishnan Parthasarathi
669783f875 Purge stale object cache entry (#2770) 2016-09-23 19:55:28 -07:00
Krishnan Parthasarathi
27e474b3d2 Improve code coverage in bucket-notification-handlers.go (#2759)
* Fix incorrect test cases for bucket-notification handler

* Add tests covering failure cases for bucket notification
2016-09-23 13:32:51 -07:00
Krishna Srinivas
1e53316241 Add tests for presigned-get (#2767)
* web-handlers: support for presigned-get json-rpc call for MinioBrowser's "share" feature.

* Add tests for presigned-get
2016-09-23 01:25:49 -07:00
Harshavardhana
ca5ca8332b bucket: refactor policies and fix bugs related to enforcing policies. (#2766)
This patch also addresses the problem of double caching at
object layer once at XL and another at handler layer.
2016-09-22 23:47:48 -07:00
Bala FA
aa579bbc20 web: add method to get all policies for given bucket name. (#2756)
Refer #1858
2016-09-22 23:06:45 -07:00
Harshavardhana
e375d822da bucket: SetBucketPolicy should save a valid Version and validate. (#2762) 2016-09-22 22:27:21 -07:00
astaxie
4bc2eb9a4d chinese translate (#2637) 2016-09-22 20:58:10 -07:00
Anis Elleuch
fc783f8407 More tests for web handlers (#2755)
* Return negative values of Total and Free in StorageInfo() when we fail to get disk info

* Return consistent messages in web handlers when the server is not initialized
2016-09-22 16:35:12 -07:00
Anis Elleuch
79e0c52fc2 Increment golang version for docker images (#2761) 2016-09-22 16:32:54 -07:00
Anis Elleuch
ef22330563 Require go 1.7.1 to build Minio server (#2727) 2016-09-22 10:33:52 -07:00
Harshavardhana
e1adbf83d8 docker: Compose config should point to edge tag. 2016-09-22 09:55:45 -07:00
Karthic Rao
1148f95292 ineffassign fixes (#2758) 2016-09-21 23:03:54 -07:00
Karthic Rao
f7430ec09c use runtime/debug.Stack() in leak detect test (#2757) 2016-09-21 22:04:35 -07:00
Karthic Rao
b8903d842c api/complete-multipart: fixes and tests. (#2719)
* api/complete-multipart: tests and simplification.

- Removing the logic of sending white space characters.
- Fix for incorrect HTTP response status for certain cases.
- Tests for New Multipart Upload and Complete Multipart Upload.

* tests: test for Delelete Object API handler
2016-09-21 20:08:08 -07:00
Aditya Manthramurthy
32f097b4d6 Controller rpc tests (#2709)
* Test code for controller-handler operations:

* Heal operations
* List operation
* Switch to "testing" lib, moving away from gocheck
* Minor refactors

* Remove extra call to initGracefulShutdown

* Remove dead code in mainControl:

Dead code found by the TestControlMain() test function that always
passes.

* Add tests for control-*-main.go
2016-09-21 19:58:50 -07:00
Krishnan Parthasarathi
559ad38b8c Add bucket-notification-handler tests (#2750) 2016-09-21 17:41:34 -07:00
Anis Elleuch
90417d2dd6 Check for bucket existence in Set/Get/Remove bucket policy workflow + tests (#2745) 2016-09-21 16:38:50 -07:00
Anis Elleuch
e66fb4bd7b configMigrate() returns errors + tests (#2735) 2016-09-21 09:44:57 -07:00
Harshavardhana
018c90dae7 events: ElasticSearch doesnt support objects with '/' in them. (#2747)
Fix this by using a unique sha256 generated for each unique key.
2016-09-20 16:36:18 -07:00
Anis Elleuch
a5066e8f76 Better code coverage of handler-utils.go (#2746) 2016-09-20 10:40:46 -07:00
Harshavardhana
0a3448c8b6 events: Change event notifiers to delete and update keys. (#2742)
ElasticSearch and Redis are both treated like a database.
Each indexs are based on the object names uniquely indentifying
the event. Upon each delete event of the named object deletes
the index on elasticsearch and redis respectively.
2016-09-20 02:11:17 -07:00
Harshavardhana
c4964232eb config: Fail to start for config mistakes. (#2740) 2016-09-19 15:23:49 -07:00
Harshavardhana
ef3c807b4a policies: Parser should handle Principals with various forms. (#2733)
Handles cases for these three combinations

  - "Principal": "*",
  - "Principal": { "AWS" : "*" }
  - "Principal": { "AWS" : [ "*" ]}

Fixes #2732
2016-09-19 13:52:28 -07:00
Harshavardhana
113b93346b lock: Make some cleanup and moving the code around. (#2718)
This patch just avoids lot of ifs and inverts some logic.
2016-09-19 13:14:55 -07:00
Krishna Srinivas
a955676986 Signature-V4: Dump the request with error message on signature mismatch. (#2734)
fixes #2691
2016-09-19 10:17:46 -07:00
Harshavardhana
725df557b5 tests: Add tests for bucket-notification-utils (#2726)
Part - 2 final fix #2711
2016-09-17 03:19:39 -07:00
koolhead17
250ac644d6 docs: Modified README.md with alias addition and bucket creation steps. (#2725) 2016-09-17 02:57:36 -07:00
Harshavardhana
490056eee3 tests: Add tests for bucket-notification-utils (#2723)
Part fix - 1 for #2711
2016-09-16 17:26:27 -07:00
Harshavardhana
797d749322 tests: Add tests for filterRuleMatch (#2722)
Part-1 fix for #2418
2016-09-16 16:44:44 -07:00
Harshavardhana
79888bfff7 tests: Add auth-handler. (#2721)
Fixes #2658
2016-09-16 15:17:49 -07:00
koolhead17
6ca57e81f1 docs: Fixed markdown typo in README.md. (#2720) 2016-09-16 14:26:28 -07:00
Anis Elleuch
010f61e91f Add more tests for event-notifier code (#2716) 2016-09-16 14:26:05 -07:00
Harshavardhana
9216981262 tests: Add test for diskCount. (#2717)
Fixes #2312
2016-09-16 13:44:52 -07:00
Anis Elleuch
b89a1cd482 tests: Implemented more tests for fs-v1*.go (#2686) 2016-09-16 13:06:49 -07:00
Harshavardhana
7d37dea449 tests: Add more streaming signature tests. (#2713)
Part fix for #2621
2016-09-16 02:45:42 -07:00
Frank
df2ef64d20 Upgrade to new dsync version incl. stale lock detection (#2708) 2016-09-16 00:30:55 -07:00
Anis Elleuch
7a549096de XL and FS use different tree walk ignored errors (#2707) 2016-09-15 13:43:40 -07:00
Harshavardhana
a1ff351f21 tests: Fix ListMultipartUploadsHandler tests. (#2705) 2016-09-15 01:44:19 -07:00
Harshavardhana
03430d0db8 tests: Add ListBucketHandler tests. (#2701)
part-3 final fix for #2412
2016-09-14 23:53:42 -07:00
Anis Elleuch
6f73d597e0 Fix tracing twice an error in fs Complete Multipart Upload (#2703) 2016-09-14 21:24:54 -07:00
koolhead17
e273a40345 docs: Modified README.md by providing information about Minio server (#2704)
data directory.
2016-09-14 21:24:12 -07:00
Anis Elleuch
a84548d7ea Fix FS remove bucket regression bug (#2693) 2016-09-14 16:41:39 -07:00
Kartik Lunkad
19e01ceb19 QuickStart docs for Minio Server Setup needs update #2698 (#2700) 2016-09-14 15:16:59 -07:00
Harshavardhana
1e6d67b16d server: Remove deadcode. (#2699) 2016-09-14 13:43:08 -07:00
Aditya Manthramurthy
6533927237 Lock-free rate-limit algorithm + bug-fix (#2694) 2016-09-14 11:27:37 -07:00
Harshavardhana
da9ae574df server: We should fail properly during server startup. (#2689)
Fixes #2688
2016-09-14 01:11:03 -07:00
Harshavardhana
16e4a7c200 Merge pull request #2657 from minio/distributed
Distributed XL support
2016-09-13 22:34:49 -07:00
Harshavardhana
ee7e70c992 tests: Add tests for ListMultipartUploads, DeleteMultipleObjects. (#2649)
Additionally adds PostPolicyHandler tests.
2016-09-13 21:22:31 -07:00
Krishna Srinivas
54a9f59a13 Init: Print SQS ARNs after globalEventNotifier is inited. (#2682)
fixes #2681
2016-09-13 21:18:30 -07:00
Harshavardhana
e6fd664331 tests: Fix format-config tests. 2016-09-13 21:18:30 -07:00
Karthic Rao
b247ec9352 tests: refactor object-handler tests. (#2656)
- Move the initialization to a common executor for object Layer API
  tests.d
2016-09-13 21:18:30 -07:00
Harshavardhana
43befab8ef Change distributed server wording. 2016-09-13 21:18:30 -07:00
Harshavardhana
eae0281c64 tests: Add GetBucketLocation, HeadBucket tests. (#2644) 2016-09-13 21:18:30 -07:00
Karthic Rao
8bd78fbdfb performance: gjson parsing for readXLMeta, listParts, getObjectInfo. (#2631)
- Using gjson for constructing xlMetaV1{} in realXLMeta.
- Test for parsing constructing xlMetaV1{} using gjson.
- Changes made since benchmarks showed 30-40% improvement in speed.
- Follow up comments in issue https://github.com/minio/minio/issues/2208
  for more details.
- gjson parsing of parts from xl.json for listParts.
- gjson parsing of statInfo from xl.json for getObjectInfo.
- Vendorizing gjson dependency.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
66459a4ce0 Add unit-tests for formatting disks during initialization (#2635)
* Add unit-tests for formatting disks during initialization

- Fixed corresponding code at places where it was deviating from the
  tabular spec.

* Added more test cases and simplified algo

... based on feedback from ``go test -coverprofile``.
2016-09-13 21:18:30 -07:00
Harshavardhana
182109f0de xl: Heal format.json properly on fresh disks. 2016-09-13 21:18:30 -07:00
Harshavardhana
9998e9ea19 api: Response timeFormat do not need to have nano-second precision.
Fixes an error reported by s3verify.
2016-09-13 21:18:30 -07:00
Harshavardhana
ba2ba328da server: Fixes for various conditions
- Fix distributed branch to be able to run FS version.
- Fix distributed branch to be able to run XL local disks.
- Ignore initialization failures of notification and bucket
  policies, the codepath should load whatever is possible.
2016-09-13 21:18:30 -07:00
Anis Elleuch
67b8080144 Fix control lock rpc name in control lock cmd (#2627) 2016-09-13 21:18:30 -07:00
Anis Elleuch
239a34ca97 Add tests for regular and streaming v4 PutObject Handler (#2618) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
81d8263ae2 binary-update: Do not fetch update info for minio binary compiled from source.
fixes #2494
2016-09-13 21:18:30 -07:00
Krishna Srinivas
b4e4846e9f PutObject: object layer now returns ObjectInfo instead of md5sum to avoid extra GetObjectInfo call. (#2599)
From the S3 layer after PutObject we were calling GetObjectInfo for bucket notification. This can
be avoided if PutObjectInfo returns ObjectInfo.

fixes #2567
2016-09-13 21:18:30 -07:00
Krishna Srinivas
92e49eab5a FS/Multipart: Do not rename append files to another tmp file as the append files are already in tmp location. (#2612) 2016-09-13 21:18:30 -07:00
Harshavardhana
c4a7b950a0 fs: Fix asynchronous multipart bug.
Construct part path properly.
2016-09-13 21:18:30 -07:00
Karthic Rao
1ce339abeb Fixing ineffssaign errors (#2608) 2016-09-13 21:18:30 -07:00
Aditya Manthramurthy
a1f922315b Add docker-compose file to run Minio in distributed mode (#2606)
Serves as a starting point to run a Minio cluster using Docker. The
file can be used as configuration for the docker-compose tool to start
4 Minio servers in distributed mode.

* Add a docker-compose.yml file to run 4 minio server instances in
  distributed mode

* Update Docker.md with command to use the file
2016-09-13 21:18:30 -07:00
Anis Elleuch
3e284162d7 Add global flags to all commands and subcommands (#2605) 2016-09-13 21:18:30 -07:00
Anis Elleuch
ff99392102 Enhance minio server help template (#2603) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
9358ee011b logging: Print stack trace in case of errors.
fixes #1827
2016-09-13 21:18:30 -07:00
Harshavardhana
37cbcae6ba xl: Remove an unecessary lock with isBucketExist() (#2593)
Fixes #2566
2016-09-13 21:18:30 -07:00
Harshavardhana
ae64b7fac8 XL: Handle object layer initialization properly.
Initialization when disk was down the network disk
reported an incorrect error rather than errDiskNotFound.

This resulted in incorrect error handling during
prepInitStorage() stage.

Fixes #2577
2016-09-13 21:18:30 -07:00
Anis Elleuch
d936ed90ae Avoid testing on system errors strings in posix (#2583) 2016-09-13 21:18:30 -07:00
Krishna Srinivas
7cc77eba45 XL/Healing: errDiskNotFound is the only pardonable error in xlShouldHeal. (#2586)
This is so that we try to heal a file for all the "bad" cases except when the disk is down.
2016-09-13 21:18:30 -07:00
Karthic Rao
07d232c7b4 instrumentation: instrumentation for locks. (#2584)
- Instrumentation for locks.
- Detailed test coverage.
- Adding RPC control handler to fetch lock instrumentation.
- RPC control handlers suite tests with a test RPC server.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
de67bca211 Move formatting of disks out of object layer initialization (#2572) 2016-09-13 21:18:30 -07:00
Anis Elleuch
5c4dbc966f Add Howto setup minio distributed with Docker 2016-09-13 21:18:30 -07:00
Harshavardhana
bca1385683 lock: Fix support single node XL locking as well. 2016-09-13 21:18:30 -07:00
Harshavardhana
f655592ff5 vendorize new dsync with new changes of logging. 2016-09-13 21:18:30 -07:00
Harshavardhana
cbe87cb2ed Fix fd-leak in rpcClient close it pro-actively. 2016-09-13 21:18:30 -07:00
Anis Elleuch
0513b3ed07 Add Heal Disk Metadata RPC API + tests (#2556) 2016-09-13 21:18:30 -07:00
Frank
7f92165c79 Single use DRWMutex, RPCClient refactor and added missing cases to lock-rpc-server (#2560)
This PR contains various fixes for the distributed release:
- Use DRWMutex in namespace-lock only for a single Lock()/RLock() call in conformance to server-side rw-locking as implemented in minio/dsync
- Implement missing cases in lock-rpc-server to catch Unlock() for active read locks and RUnlock() for an active write lock
- Refactor RPCClient to release local mutex while making actual RPC.Call()
2016-09-13 21:18:30 -07:00
Harshavardhana
780ccc26f7 server: Validate server arguments for duplicates. (#2554)
- Validates invalid format inputs.
- Validates duplicate entries.
- Validates sufficient amount of disks.

Partially fixes #2502
2016-09-13 21:18:30 -07:00
Harshavardhana
339425fd52 server: Fetch StorageInfo() from underlying disks transparently. (#2549)
Fixes #2511
2016-09-13 21:18:30 -07:00
Harshavardhana
fa6e9540a8 server: We shouldn't exit the server in lazy init. (#2548)
Avoid fatalIf instead these are non-critical errors,
continue running the server.

 - initializing bucket notifications
 - initializing bucket policies.
 - migrating bucket policies failure.

Fixes #2547
2016-09-13 21:18:30 -07:00
Harshavardhana
9605fde04d controller/auth: Implement JWT based authorization for controller. (#2544)
Fixes #2474
2016-09-13 21:18:30 -07:00
Anis Elleuch
200d327737 List only objects that need healing (#2546) 2016-09-13 21:18:30 -07:00
Harshavardhana
e1b0985b5b rpc: Refactor authentication and login changes. (#2543)
- Cache login requests.
- Converge validating distributed setup.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c8dfc4cda4 auth-rpc: Reset token on disconnect (#2542) 2016-09-13 21:18:30 -07:00
Bala FA
7922a54c9a rpc-client: remove unwanted nil check of rpcClient. (#2538) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
bda6bcd5be Layered rpc-client implementation (#2512) 2016-09-13 21:18:30 -07:00
Harshavardhana
7e3e24b394 rpc: client login should ignore server versions. 2016-09-13 21:18:30 -07:00
Harshavardhana
bb0466f4ce control: Fix controller CLI handling with distributed server object layer.
Object layer initialization is done lazily fix it.
2016-09-13 21:18:30 -07:00
Harshavardhana
8797952409 server: Add server command line for running in distributed mode 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
c33d1b8ee6 Vendorize minio/dsync for server-side read lock (#2484)
- Prevention of stale lock accumulation.
- Removal of dead code.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
1f67c18222 Lock/Rlock rpc reply was incorrect (#2479) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
a4691611a7 Move initGracefulShutdown before objectLayer initialization (#2468) 2016-09-13 21:18:30 -07:00
awwalker
7c7eb1475d splitNetPath: Add support for windows paths including volumeNames e.g ip:C:\network\path 2016-09-13 21:18:30 -07:00
Harshavardhana
0bce3d6d63 Fix web handler initialize with distributed lazy init. 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
804d91ef61 storage/rpc-client: Reconnect on network disconnect (#2436) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
229600ce9b Implement RLock, RUnlock rpc handlers (#2437)
This would make it simplify dsync.RWMutex's algorithm to acquire a
distributed read lock and be tolerant to N/2-1 failures.
2016-09-13 21:18:30 -07:00
Harshavardhana
43098df9d8 rpc: Re-factor ReadFile behavior client <--> server.
Current code did not marshal/unmarshal buffers properly from
the server, The reason being buffers have to be allocated and
sent properly back to client to be consumable.
2016-09-13 21:18:30 -07:00
Harshavardhana
6908a0dcd4 Extract rpc server wrapped errors and translate to storage error. 2016-09-13 21:18:30 -07:00
Harshavardhana
cae5761f16 rpc/client: Add missing rpcTokens for each rpc calls. 2016-09-13 21:18:30 -07:00
Harshavardhana
83074ed57e Add missing rpc-client.go - missing in previous commit. 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
64a7e6992e Vendorize rpc reconnect changes from minio/dsync (#2405) 2016-09-13 21:18:30 -07:00
Harshavardhana
61af764f8a Add rpc layer authentication. 2016-09-13 21:18:30 -07:00
Harshavardhana
b4172ad3c8 Rename rpc-{client,server} storage-rpc-{client,server} 2016-09-13 21:18:30 -07:00
Harshavardhana
4917038f55 Move the ObjectAPI() resource to be beginning of each handlers.
We should return back proper errors so that the clients can
retry until server has been initialized.
2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b0f3f94163 unify single-node and distributed namespace locking (#2401) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
b7c169d71d object layer initialization using distributed locking (#2397)
* vendorized latest minio/dsync

* wip - object layer initialization using distributed locking
2016-09-13 21:18:30 -07:00
Frank
3939c75345 Added distributed RWMutex (#2369) 2016-09-13 21:18:30 -07:00
Krishnan Parthasarathi
e55926e8cf distribute: Make server work with multiple remote disks
This change initializes rpc servers associated with disks that are
local. It makes object layer initialization on demand, namely on the
first request to the object layer.

Also adds lock RPC service vendorized minio/dsync
2016-09-13 21:18:30 -07:00
Anis Elleuch
f82f535509 Fix possible race in shutdown callbacks process and simplify shuttting down profiler (#2684) 2016-09-13 18:43:45 -07:00
Anis Elleuch
51e337228e Avoid hiding disk errors in some cases in FS Shutdown (#2668) 2016-09-13 11:01:10 -07:00
Kevin Qiu
241c56e6d7 Use Set instead of Add in the event that the request already contains the content-length (#2683) 2016-09-13 10:59:40 -07:00
Aditya Manthramurthy
895471afa1 Fixes #2678 (#2679)
Refactor `ratelimit.acquire()` to properly enforce the *globalMaxConn*
limit.
2016-09-12 15:53:54 -07:00
Harshavardhana
85e2d886bc bucketPolicy: Trim incoming paths properly for policy verification. 2016-09-11 10:36:54 -07:00
Harshavardhana
07aa02f196 build: Fix appveyor.yml to report windows coverage as well. (#2666) 2016-09-10 21:05:00 -07:00
Anis Elleuch
8acf4d112a Return Owner information in ListObjectV2 only when fetch-owner is specified (#2654) 2016-09-10 10:44:38 -07:00
Anis Elleuch
e6abfb3b67 More tests for format-config for a better coverage (#2655) 2016-09-10 09:54:22 -07:00
Harshavardhana
040af08473 server: Startup message now prints configured ARNs. (#2653)
Fixes #2628
2016-09-10 02:23:28 -07:00
Harshavardhana
e3de237eb8 Move config to v7 to fix previous v6 bug on null Notification entries. (#2650)
Thanks to @koolhead17 for reporting this.
2016-09-10 00:51:25 -07:00
Anis Elleuch
11785529fc Add Request Not Ready error when presigned date request is not valid (#2646) 2016-09-10 00:38:07 -07:00
Anis Elleuch
32201a18ab Forbid requests generated at least 15 min in the past or in the future (#2648) 2016-09-09 15:33:35 -07:00
Anis Elleuch
421cccb1d7 Add naughty disk to simulate disk errors (#2645) 2016-09-09 12:53:09 -07:00
Krishna Srinivas
400e9309f1 Signature-V4: Include content-length for signature calculation. (#2643)
This is to be compatible with clients which includes content-length for signature calculation
(and hence deviating from AWS Signature-v4 spec)
2016-09-09 09:38:07 -07:00
Krishna Srinivas
a4afb312d4 S3-layer: Multiple-delete should return success for non-existent objects. (#2633)
Fixes #2630
2016-09-07 11:49:12 -07:00
Harshavardhana
afe874f15a api: CopyObject pipe should be closed pro-actively upon error. (#2625)
Fixes a deadlock reproduced while running s3verify during
RemoveObject().  Previously held lock by GetObject() inside
the go-routine was never relenquished.
2016-09-06 02:23:32 -07:00
Harshavardhana
b291dbe9c5 s3: Fix compliance bugs reported by s3verify. (#2624)
- Fixes couple of error strings reported are mismatching.
- Fixes a error HTTP status which was wrong fixed.
- Remove usage of an deprecated PostResponse, au contraire
  to their documentation there is no response body in
  PostPolicy.
2016-09-06 02:18:02 -07:00
Harshavardhana
bc8f34bfe7 server/mux: Remove unused waitgroup from listenerMux. (#2614)
Change struct names to be more meaningful.
2016-09-02 19:07:42 -07:00
Harshavardhana
2dc7ecc59b notifications: Fix bucket notifications for DeleteMultipleObjects. (#2609)
Now reports bucket notifications for DeleteMultipleObjects API.
Also deletes multiple objects in parallel.
2016-09-02 01:59:08 -07:00
Hugo Jobling
600551feb9 Make export directory an exposed volume (#2610)
This exposes the /export directory, making it possible to mount it from other
containers. It also makes it possible to bind a local folder to the internal /export
mount point using the native Docker for Mac/Windows and Kinematic.
2016-09-02 01:58:15 -07:00
Harshavardhana
7398d737b5 profiler: Fix it properly and generate/save profiles even failure situations. (#2607)
Fixes #2594
2016-09-01 20:13:11 -07:00
Harshavardhana
bf62ba57cf docs: Update minimum drives in erasure docs. 2016-09-01 11:35:53 -07:00
Harshavardhana
511a8cbe04 profiler: Fix how we generate profiles. (#2596)
Fixes #2594
2016-09-01 03:50:31 -07:00
koolhead17
790ad68d4d Doc: Fixed Minio install steps from source. (#2590) 2016-08-31 18:00:11 -07:00
Anis Elleuch
e79d2381fc Fix rare 'go test -race' failure in ListenServe{Plain,TLS} (#2588) 2016-08-31 16:07:44 -07:00
Krishna Srinivas
a3b4199e9b FS/Multipart: Allow for parallel uploads of parts with same part number. (#2587)
CBL client does not close connection when the backup process is stopped, this causes
read() on the stream on the server side to block and hence the lock held on the part
is not released. When the backup process is restarted, we again try to lock on the
part and this will block. Using a unique tmp name and not locking it fixes the problem.
2016-08-31 16:01:32 -07:00
Krishna Srinivas
3aa0574c6b FS/multipart: Append the parts to a file as and when the parts arrive. (#2513) 2016-08-31 13:42:57 -07:00
Andrew Stone
3118195e92 docker: Minimize image size (#2585)
This takes the image size from 823.1MB (current edge) to 281.8MB.

The following steps are taken:
  0) Use go-alpine for a much smaller base image
  1) Perform all actions in 1 step to eliminate diff images
  2) Remove unused files created after setup
2016-08-31 13:32:41 -07:00
Harshavardhana
5716f1c199 Fix incorrect error strings. (#2580) 2016-08-30 13:47:09 -07:00
Harshavardhana
2020afd200 server: http.Server do not add deadlines causes issues. (#2581)
Adding deadlines is a no go since Golang doesn't back off
the timers if there is an active i/o in progress.

It is meant to be for applications to handle this themselves
and manually progress the deadlines.

Fixes #2561
2016-08-30 12:52:19 -07:00
Bala FA
7431acb2c4 web: add handlers for set/get bucket policy. (#2486)
Refer #1858
2016-08-30 10:04:50 -07:00
Harshavardhana
2f9975c76c notifications: GetObjectInfo() later if bucket notifications are set. (#2575) 2016-08-30 03:42:37 -07:00
Dee Koder
d9bd73f4e8 docs: Simplified install instructions for home-brew. (#2571) 2016-08-29 14:08:04 -07:00
koolhead17
8484d1c0ca Doc: Modified README.md to add brew package manager install steps. (#2557) 2016-08-28 15:44:56 -07:00
Harshavardhana
7270ca4157 pkg/wildcard: Simplify the wildcard logic further. (#2555) 2016-08-27 00:27:17 -07:00
Brendan Ashworth
b67c8970c9 bucket-handlers: fix missing return statement (#2553)
If the location was invalid, it would write an error response but then
continue to attempt to make the bucket. Whether or not it would succeed,
it would attempt to call response.WriteHeaders twice in a row, which
would cause a message to be logged to the server console (bad).

Here is the relevant Go code:
c80e0d374b/src/net/http/server.go (L878-L881)
2016-08-25 20:00:47 -07:00
Mohit Agarwal
418921de89 minor cleanup
- Reused contains() from utils.go at a couple of places
- Cleanup in return statements and boolean checks
2016-08-24 22:54:34 -07:00
Harshavardhana
ec4260d260 api: BucketNotification should disallow duplicate notification. (#2539)
Added checks to look for duplicated notification configs.

Fixes #2472
2016-08-23 18:42:30 -07:00
Harshavardhana
c39d3db7a0 server/tls: allocate tls.Config{} properly. (#2537)
There is a golang bug which crashes the server, allocate tls.Config properly to avoid
this problem.

```
2016/08/22 20:21:03 http: panic serving 127.0.0.1:40744: runtime error: invalid memory address or nil pointer dereference
goroutine 38 [running]:
net/http.(*conn).serve.func1(0xc820526680)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1389 +0xc1
panic(0xbefa80, 0xc820010140)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/runtime/panic.go:443 +0x4e9
crypto/tls.(*Conn).serverHandshake(0xc820368300, 0x0, 0x0)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/handshake_server.go:43 +0x4d6
crypto/tls.(*Conn).Handshake(0xc820368300, 0x0, 0x0)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/crypto/tls/conn.go:1035 +0x169
net/http.(*conn).serve(0xc820526680)
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:1405 +0x382
created by net/http.(*Server).Serve
    /home/harsha/.gimme/versions/go1.6.2.linux.amd64/src/net/http/server.go:2137 +0x44e
```

Fixes #2536
2016-08-23 00:20:15 -07:00
Harshavardhana
2da0cfc904 signature: Use the client sent region if region is properly validated. (#2534)
A properly validated region string from the client should be once
validated properly.

Fixes #2521
2016-08-22 19:43:36 -07:00
Harshavardhana
9dd0e3dc44 config: Initialize notify for fresh configs. (#2531) 2016-08-22 16:04:04 -07:00
Dee Koder
4722c94653 typo: Fix capitalization in heading. (#2529) 2016-08-22 12:01:21 -07:00
Krishna Srinivas
45c928e2f5 boot: checkPortAvailability() should fail only for EADDRINUSE error and ignore other errors. (#2527)
fixes #2510
2016-08-22 10:20:01 -07:00
Anis Elleuch
07506358ff Refactor Heal RPC and add Shutdown RPC (#2488) 2016-08-21 12:06:53 -07:00
Harshavardhana
975eb31973 api/bucketPolicy: Use minio-go/pkg/set and fix bucket policy regression. (#2506)
Current master has a regression 'mc policy <policy-type> alias/bucket/prefix'
does not work anymore, due to the way new minio-go changes do json marshalling.
This led to a regression on server side when a ``prefix`` is provided
policy is rejected as malformed from th server which is not the case with
AWS S3.

This patch uses the new ``minio-go/pkg/set`` package to address the
unmarshalling problems.

Fixes #2503
2016-08-20 03:16:38 -07:00
koolhead17
a3c509fd23 Doc: Modified examples in how-to-run-multiple-minio-server-instances-on-single-machine.md (#2497) 2016-08-19 16:09:21 -07:00
GarimaKapoor
63bb78cfc6 Update README.md 2016-08-19 11:46:03 -07:00
Jesse Lucas
f2fd8b0265 Adding test coverage for server-mux.go by creating TestListenAndServeTLS(). (#2493)
Editing config.go to add lock on customConfigPath to avoid race condition
when setGlobalConfigPath() and getConfigPath() are called concurrently.
2016-08-19 01:29:50 -07:00
Jesse Lucas
a8052889fe server-mux_test.go updating test to fix race condition with TestListenAndServe (#2485) 2016-08-18 19:45:03 -07:00
Harshavardhana
bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00
Anis Elleuch
73d1a46f3e Allow asterisk character in key names (#2487) 2016-08-18 12:00:01 -07:00
Harshavardhana
95c16f51cb notification: Handle ARN with empty account and name types. (#2482)
Fixes #2481
2016-08-17 16:34:03 -07:00
Harshavardhana
810dcbf34b fs: ListObjects should populate ETag properly if fs.json is available. (#2480)
Fixes #2470
2016-08-17 13:26:08 -07:00
Harshavardhana
cb77586508 controller: Fix controller cli arguments. 2016-08-17 11:41:04 -07:00
Krishna Srinivas
e2498edb45 contoller: Implement controlled healing and trigger (#2381)
This patch introduces new command line 'control' 

- minio control 

TO manage minio server connecting through GoRPC API frontend.

- minio control heal 

Is implemented for healing objects.
2016-08-17 11:36:33 -07:00
Jesse Lucas
0b7dfab17a Refactoring MuxServer Close() method to always wait for conns to close before returning. Adding lock around ServerMux listener setting to protect against data race. Adding additional tests to server-mux_test.go to make sure open connections are closed and for ListenAndServe. (#2467) 2016-08-17 00:18:23 -07:00
Krishna Srinivas
674fdc4304 redirect-middleware: Redirect to /minio only if client is a browser and the request is not V4 signed. (#2469)
fixes #2356
2016-08-17 00:16:59 -07:00
karthic rao
10feb1af3f tests: object handlers: Unit tests for Get and Copy Object handlers (#2451) 2016-08-16 19:24:23 -07:00
GarimaKapoor
d2b924cca8 Added Goreportcard badge. 2016-08-16 17:59:59 -07:00
Brendan Ashworth
13390d0c95 tests: add tests for signature-v4.go (#2458)
This adds tests for the functions `doesPresignedSignatureMatch` and
`doesPolicySignatureMatch`.
2016-08-16 17:02:35 -07:00
Harshavardhana
0a3d43273f vendor: sha256 32bit updated. (#2459) 2016-08-16 16:19:29 -07:00
Harshavardhana
4dec50ba51 build: Add platform specific fixes. 2016-08-16 14:40:41 -07:00
Krishna Srinivas
f2bffe6086 fs/delete-object: fs.json filepath was incorrect. (#2448) 2016-08-16 10:05:08 -07:00
Krishna Srinivas
8e2f64aea4 fs/multipart: save metadata(fs.json) for multipart uploads. (#2450) 2016-08-16 10:04:40 -07:00
Harshavardhana
c054e633fd utils: Shutdown channel should be bufferred. 2016-08-15 21:01:24 -07:00
Harshavardhana
e86dfcf41e api: Change listen bucket notification to be TopicConfiguration. (#2447) 2016-08-15 20:56:43 -07:00
Anis Elleuch
3b9dbd748b tests: Web handlers (#2429) 2016-08-15 16:13:03 -07:00
Harshavardhana
3d1bb8f439 tests: Fix hasExtendedHeader tests with env variable. 2016-08-15 16:09:08 -07:00
Krishna Srinivas
bb8a425d49 When updating the meta file, write to temp file first and then rename to the actual location.
This prevents appending the metadata to the metadata-file when a file is reuploaded.
2016-08-15 15:55:59 -07:00
Harshavardhana
0e745fdb05 fs: Enable fs.json with env MINIO_ENABLE_FSMETA 2016-08-15 15:53:48 -07:00
Anis Elleuch
51d7749c3e Check if eventN is initialized before notifying in Upload web handler (#2435) 2016-08-15 12:15:46 -07:00
Harshavardhana
76d56c6ff2 typo: Fix typos across the codebase. (#2442) 2016-08-15 02:44:48 -07:00
Harshavardhana
b41bfcbf2f utils: Fix unit tests issue. (#2441) 2016-08-15 01:59:28 -07:00
Yurii Rashkovskii
341171f326 Problem: AWS documentation defines event timestamp as 1970-01-01T00:00:00.000Z (#2440)
While Minio is using 20160814T124605Z

(See http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html)

Solution: adhere to AWS documentation
2016-08-15 01:50:07 -07:00
karthic rao
a3592228f5 bug-fix: fix for tests failure when cache is disabled (#2439) 2016-08-15 01:25:41 -07:00
Anis Elleuch
5526ac13d2 Protect shutdown callbacks lists with a mutex (#2432) 2016-08-14 23:55:48 -07:00
Harshavardhana
9606cb9bcd posix: Disk free verification should have relaxed handling of inodes. (#2431)
Some filesystems do not implement a way to provide total inodes available, instead inodes
are allocated based on available disk space. For example CephFS, StoreNext CVSFS, AzureFile
driver. Allow for the available disk to be separately validate and we will validate inodes
only if the total inodes are provided by the underlying filesystem.

Fixes #2364
2016-08-13 02:30:15 -07:00
Yurii Rashkovskii
7829ccea2c Routing key was misspelled as routine key (#2430) 2016-08-12 22:23:06 -07:00
Anis Elleuch
723153951c Test api responses (#2427)
* Test List Multipart Uploads with correct max-keys

* Test List Objects V1 V2 with valid parameters
2016-08-12 11:28:27 -07:00
Anis Elleuch
64dc2a2e7f Heal format by inspection should avoid picking minioMetaBucket dir if the latter comes first in the list (listDir not ordered) (#2426) 2016-08-12 08:38:04 -07:00
Anis Elleuch
fdab984c8d Add test for fresh disks healing (#2424) 2016-08-12 08:36:43 -07:00
karthic rao
70fd38818e clean up: ineffassign fixes (#2411) 2016-08-12 00:26:30 -07:00
Jesse Lucas
ef0a108dde Graceful shutdown for ServerMux (#2341) 2016-08-11 21:33:55 -07:00
karthic rao
0b225269e1 tests: posix: tests cleaning up and enhancing coverage. (#2410) 2016-08-11 19:57:14 -07:00
Anis Elleuch
fe62688683 Add tests for Damerau Levenshtein algorithm (#2407) 2016-08-11 17:50:04 -07:00
Anis Elleuch
fadb71351c Test Post policy parsing and checking conditions (#2408) 2016-08-11 17:49:40 -07:00
Matthieu Fronton
402af93da2 Update how-to-install-golang URL (#2406) 2016-08-11 12:01:12 -07:00
Harshavardhana
d1bb8a5b21 api: refactor the bucket policy reading and writing. (#2395)
Policies are read once during server startup and subsequently
managed through in memory map. In-memory map is updated as
and when there are new changes coming in.
2016-08-10 20:10:47 -07:00
Harshavardhana
97c1289659 handlers: GetObject and HeadObject support more responses. (#2403)
- response-content-encoding.
- response-content-language.

Fixes #2393
2016-08-10 17:36:28 -07:00
Harshavardhana
8274ac2e5a tests: Make sure we try tests on free ports. (#2402)
Fixes #2376
2016-08-10 16:28:05 -07:00
Brendan Ashworth
758aa21b9c tests: add tests for certs.go and trie.go (#2394) 2016-08-10 02:26:40 -07:00
koolhead17
0dc5662f9b Doc: Fixed screenshot path for modified docs. (#2390) 2016-08-09 15:04:44 -07:00
Harshavardhana
82cd38e959 handlers: Remove 'notification.xml' when bucket is deleted. (#2389)
Do not pass around objectHandlers object, input argument
should comply to a type for only that would be used inside
the function body.
2016-08-09 11:33:45 -07:00
karthic rao
e0cf4ee9fc presignV4: fix errors response and tests. (#2375)
- Fix error response when one of the query params in the presign URL is
  missing.

- Exhasutive test coverage for presignv4.
2016-08-09 09:13:15 -07:00
Dee Koder
2a920e568c docs: Readded code coverage badge in github Readme. (#2391)
We have the fix in place to hide this on docs.minio.io
2016-08-08 22:14:38 -07:00
Harshavardhana
9c7f75d1e7 handler: Remove unused accesslog handler (#2388) 2016-08-08 21:33:21 -07:00
Harshavardhana
7e46055a15 api/handlers: Implement streaming signature v4 support. (#2370)
* api/handlers: Implement streaming signature v4 support.

Fixes #2326

* tests: Add tests for quick/safe
2016-08-08 20:56:29 -07:00
koolhead17
0c125f3596 Doc: This patch new guides with titles mentioned below (#2382) 2016-08-08 19:39:01 -07:00
GarimaKapoor
a1f3bf57c7 Update README.md 2016-08-07 11:08:42 -07:00
Harshavardhana
0188cd0b84 utils: Take monitorShutdownSignal to take an exitFunc which would executed upon error. (#2378)
This hook approach allows program to keep running but being able to handle exiting
of the program in the dynamic way.

Fixes #2377
2016-08-06 23:53:10 -07:00
Harshavardhana
b23605a2b5 pkg/objcache: Add more tests. (#2371) 2016-08-06 10:22:14 -07:00
koolhead17
8c2985a9f5 Doc: README.md/Removed codecov badge from title. (#2367) 2016-08-05 16:53:29 -07:00
Anis Elleuch
d28fb5fe23 Add a generic registerShutdown function for graceful exit (#2344)
* Add a generic registerShutdown function for graceful exit
* Add shutdown callback test case
2016-08-05 13:48:31 -07:00
GarimaKapoor
62c0612eac Update README.md 2016-08-05 11:35:03 -07:00
karthic rao
d63ce9d60d tests: tests for signature v4 parser (#2362) 2016-08-05 08:03:18 -07:00
Remco Verhoef
5a44c34fd7 fixed some issues in readme (#2363)
thx to @MartijnVogel_twitter
2016-08-05 08:02:11 -07:00
Harshavardhana
064c51162d api: Add new ListenBucketNotificationHandler. (#2336)
This API is precursor before implementing `minio lambda` and `mc` continous replication.

This new api is an extention to BucketNofication APIs.

// Request
```
GET /bucket?notificationARN=arn:minio:lambda:us-east-1:10:minio HTTP/1.1
...
...
```

// Response
```

{"Records": ...}
...
...
...
{"Records": ...}
```
2016-08-04 22:01:58 -07:00
Harshavardhana
90c20a8c11 Add codecov for minio. (#2359) 2016-08-04 16:48:50 -07:00
Harshavardhana
e783d77c3d Add codecov for minio. 2016-08-04 15:56:26 -07:00
Krishna Srinivas
e887fea485 getbucketlocation: should handle UNSIGNED-PAYLOAD for sha256 header for signature calculation. (#2358)
fixes #2355
2016-08-04 15:49:35 -07:00
Harshavardhana
de5d5ff241 pkg/crypto: Deprecate cgo sha256 version. (#2354) 2016-08-04 03:19:36 -07:00
karthic rao
2e0742e309 bucket policy: Support for '?' wildcard. (#2353)
- Support for '?' wildcard for resource matching.

- Wildcard package is added with Match functions.

- Wildcard.Match supports '*' and wild.MatchExtended supports both '*'
  and '?' wildcards in the pattern string.

- Tests for the same for the wide range of cases.
2016-08-04 00:41:32 -07:00
Anis Elleuch
cc0d5b6fe0 webapi: ServerInfo returns the list variables/values in the environnment of the Minio server (#2351) 2016-08-03 13:47:03 -07:00
Harshavardhana
2db51e9d61 server/config: config should be updated only when we edit the credentials. (#2345) 2016-08-02 16:48:21 -07:00
Krishna Srinivas
69fd196471 Object-cache: enforce cache size to be less than RAM. (#2338) 2016-08-02 10:04:35 -07:00
karthic rao
1494af485e tests: signature-utils test (#2342) 2016-08-01 20:54:11 -07:00
Harshavardhana
c1d70f1f9e server/config: Create 'certs' directory in initServerConfig(). (#2331)
certs directory was created only if config was not present, our
expectancy is we need 'certs' directory to be present all the
time making it easier to be documented.
2016-07-30 14:55:20 -07:00
karthic rao
9baf599c91 tests: Unit tests and fixes for copyBuffer. (#2333)
- Unit tests for copyBuffer.
- Shadowing fix for copyBuffer.
2016-07-30 13:36:43 -07:00
Harshavardhana
8d090a20ce server: set globalCacheSize honoring system limits for max memory. (#2332)
On unix systems it is possible to set max memory used by
running processes using 'ulimit -m' or 'syscall.RLIMIT_AS'.

A process whence exceeds this limit, kernel would pro-actively
kill such a server with OOM. To avoid this problem of defaulting
our cache size to 8GB we should look for if the current system
limits are lower and set the cache size appropriately.
2016-07-30 08:50:49 -07:00
karthic rao
5b86dd7659 Tests: Cleanup/Enhancement: Add few more cases to posix.ReadFile tests and use a cleaner posixTestSetup for posix tests (#2330) 2016-07-30 01:26:19 -07:00
Jesse Lucas
4b05b6a6c1 Refactoring checkPortAvailability to check each tcp network (tcp, tcp4, tcp6) if a port is taken. (#2325) 2016-07-29 18:11:00 -07:00
Jesse Lucas
851d05161a Adding return error value to checkPortAvailability to enable testing of function. Adding checkport_test.go to test checkPortAvailability. Updated server-main.go to use error value from checkPortAvailability and calls fatalIf if an error is returned. (#2322) 2016-07-29 14:05:31 -07:00
Harshavardhana
cf9ba7b88f tests: Add missing unit test for posix.ReadFile. (#2319) 2016-07-28 21:57:11 -07:00
Krishnan Parthasarathi
50dae0ab04 bucket-policy: Migrate bucket policy to minioMetaBuket/buckets (#2321) 2016-07-28 20:49:08 -07:00
Anis Elleuch
14cefd352c Heal corrupted formats of disks already containing objects (#2297) 2016-07-28 16:49:58 -07:00
Frank
f239fcac67 Switched to faster minio/sha256-simd implementation (#2320) 2016-07-28 14:44:37 -07:00
Anis Elleuch
dcc3463e48 Limit POST form fields and file size + Generic Request Size limiter (#2317)
* Use less memory when receiving a file via multipart
* Add generic http request maximum size limiter to secure against malicious clients
2016-07-28 12:02:22 -07:00
Krishna Srinivas
7850d17f48 web-browser: disable minio browser when environmental variable MINIO_BROWSER=off (#2315)
fixes #2314
2016-07-28 04:00:33 -07:00
Harshavardhana
f503ac3db8 XL/Erasure: Make bit-rot verification based on xl.json algo. (#2299)
Currently `xl.json` saves algorithm information for bit-rot
verification. Since the bit-rot algo's can change in the
future make sure the erasureReadFile doesn't default to
a particular algo. Instead use the checkSumInfo.
2016-07-28 02:20:34 -07:00
Harshavardhana
65f71ce0c5 browser: Object upload should save metadata and notify. (#2309)
Object upload from browser should save additional
incoming metadata. Additionally should also notify
through bucket notifications once they are set.

Fixes #2292
2016-07-27 21:11:15 -07:00
Harshavardhana
ad19bf0ec1 server: Add update referral for update notification URL. (#2308) 2016-07-27 19:59:19 -07:00
Harshavardhana
f0067babe0 handlers: Add 'crossdomain.xml' handler. (#2305)
Fixes #2301
2016-07-27 19:53:55 -07:00
karthic rao
6b5fce826b placing defer file.Close() right after opening it (#2306) 2016-07-27 19:22:32 -07:00
Anis Elleuch
8b3cb3a0de POST Policy, multiple fixes: AccessDenied with unmet conditions, ${filename} in Key, missing filename in multipart (#2304)
* Unsatisfied conditions will return AccessDenied instead of MissingFields

* Require form-field `file` in POST policy and make `filename` an optional attribute

* S3 feature: Replace  in Key by filename attribute passed in multipart
2016-07-27 17:51:55 -07:00
Harshavardhana
2f7358a8a6 XL: erasure Index should have its corresponding distribution order. (#2300) 2016-07-27 11:57:08 -07:00
Harshavardhana
77248bd6e8 api: Notify events only if bucket notifications are set. (#2293)
While the existing code worked, it went to an entire cycle
of constructing event structure and end up not sending it.

Avoid this in the first place, but returning quickly if
notifications are not set on the bucket.
2016-07-26 19:10:02 -07:00
Harshavardhana
3054b74260 docs: Fix startup message for server as well. 2016-07-26 15:54:11 -07:00
koolhead17
7d42d09da8 Doc: Replaced README & FreeBSD docs with updated minio server splash (#2298)
screen.
2016-07-26 15:46:41 -07:00
Anis Elleuch
95ddf061ab Rate limit is working and supports limited waiting clients (#2295) 2016-07-26 14:17:11 -07:00
karthic rao
5fe72cf205 Removing readAllMeta from xl-v1-healing.go and placing it in xl-v1-utils.go (#2296) 2016-07-26 11:34:48 -07:00
Harshavardhana
1e3d80552f XL: format.json healing should cater for mismatching order. (#2285)
Fresh disks can be provided in any order, we need to make sure
to preserve existing disk order and populate the fresh disks
in new positions.

Thanks for Anis Elleuch <vadmeste@gmail.com> for finding this issue.
2016-07-26 03:18:47 -07:00
Harshavardhana
f253dfc922 docs: Fix erasure code image embedding issue. 2016-07-26 00:09:57 -07:00
Harshavardhana
1f9e38e3cd api: Add bucket notification util tests. (#2289) 2016-07-26 00:01:35 -07:00
karthic rao
530ed67b59 Adding leak test framework (#2254) 2016-07-25 20:39:14 -07:00
Harshavardhana
a2b6f0524d XL/erasure: Remove deprecated copyN function. (#2288) 2016-07-25 20:36:56 -07:00
karthic rao
091d80666a Enhancement for Erasure encode test. (#2287) 2016-07-25 20:36:41 -07:00
Harshavardhana
efbf7dbc0f api: Bucket notification add filter rules check and validate. (#2272)
These filtering techniques are used to validate
object names for their prefix and suffix.
2016-07-25 17:53:55 -07:00
Krishna Srinivas
043ddbd834 optimize memory allocation during erasure-read by using temporary buffer pool. (#2259)
* XL/erasure-read: optimize memory allocation during erasure-read by using temporary buffer pool.

With the change the buffer needed during GetObject by erasureReadFile is allocated only once.
2016-07-25 14:17:01 -07:00
Dee Koder
04f90bd463 doc: Broken links fixed in the Explore further section. (#2281) 2016-07-24 22:53:35 -07:00
Harshavardhana
9212e11b90 XL/GetObject: When disk is not available, checksum should be empty. (#2276) 2016-07-24 22:49:27 -07:00
Harshavardhana
79bab6b561 XL: Operations on uploads.json should cater for disk being unavailable. (#2277) 2016-07-24 18:08:15 -07:00
Krishnan Parthasarathi
7e5a78985d tests: Using listObjects clean up remaining tree walk go routines. (#2278)
* fs: Set nextMarker independent of it having a slash or not.
* tests: Using listObjects clean up remaining tree walk go routines.
* tests: Use slices to hold data instead of enumerating test cases by hand

... also fixed numbering of test cases.
2016-07-24 15:52:12 -07:00
Anis Elleuch
b0b919a1d6 Server http and https on the same port using a customized server (#2247) 2016-07-24 12:30:57 -07:00
Harshavardhana
6c2fb19ed7 docs: Removed and purged uneeded docs. (#2273) 2016-07-24 03:32:45 -07:00
Harshavardhana
f248089523 api: Implement bucket notification. (#2271)
* Implement basic S3 notifications through queues

Supports multiple queues and three basic queue types:

1. NilQueue -- messages don't get sent anywhere
2. LogQueue -- messages get logged
3. AmqpQueue -- messages are sent to an AMQP queue

* api: Implement bucket notification.

Supports two different queue types

- AMQP
- ElasticSearch.

* Add support for redis
2016-07-23 22:51:12 -07:00
Harshavardhana
f85d94288d api: extract http headers with some supported header list. (#2268) 2016-07-22 20:31:45 -07:00
Harshavardhana
55cb55675c api/multipart: Send S3 compatible error message, missing second sentence. (#2270) 2016-07-22 17:05:40 -07:00
koolhead17
7e076577de Update Minio-erasure-code-quickStart-guide.md (#2269)
Minor update to link the URL.
2016-07-22 13:35:27 -07:00
Harshavardhana
5d118141cd XL: Remove deadcode unionChecksumInfo. (#2261) 2016-07-21 19:07:00 -07:00
karthic rao
646ff2c64d Get Object disk not found test (#2264)
Test: GetObject disk not found test
2016-07-21 19:06:50 -07:00
Harshavardhana
0add96f655 fs: Save metadata for objects in minioMetaBucket directory. (#2251) 2016-07-21 17:31:14 -07:00
Krishna Srinivas
303f216150 tests: xl-v1-metadata.go, xl-v1-multipart-common.go - remove unused methods, add enhance tests to improve code coverage. (#2260) 2016-07-21 15:00:11 -07:00
koolhead17
a7b5b8e63f Doc: Modified the contents for Doctor. (#2262) 2016-07-21 14:58:16 -07:00
Krishnan Parthasarathi
5730d40478 tests: Added GetObject, DeleteObject and PutObject unit-tests (#2222) 2016-07-21 13:15:54 -07:00
karthic rao
0eaf684777 Remove consuming benchmarks, clean up closures, correct Get and PutObject Parallel benchmarks (#2258) 2016-07-21 11:17:28 -07:00
Harshavardhana
a0635dcdd9 XL: Do not rely on getLoadBalancedQuorumDisks for NS consistency. (#2243)
The reason is any function relying on `getLoadBalancedQuorumDisks`
cannot possibly have an idempotent behavior.

The problem comes from given a set of N disks returning just a
shuffled N/2 disks.  In case of a scenario where we have N/2
number of failed disks, the returned value of `getLoadBalancedQuorumDisks`
is not equal to the same failed disks so essentially calls using such
disks might succeed or fail randomly at different intervals in time.

This proposal change is we move to `getLoadBalancedDisks()`
and use the shuffled N disks as a whole. Since most of the time we might
hit a good disk since we are not reducing our solution space. This
also provides consistent behavior for all the functions which rely
on shuffled disks.

Fixes #2242
2016-07-21 00:27:08 -07:00
Dee Koder
41f4f2806d screenshots: update with the latest optimized image. (#2249) 2016-07-20 16:15:26 -07:00
Dee Koder
2a972ef1fd images: Move screenshot for docs inside docs/screenshots directory. (#2248)
* images: Move screenshot for docs inside docs/screenshots directory. Use optimized images.

* images: This fix optimizes the images for the Erasure Code Quick Start Guide
2016-07-20 13:52:30 -07:00
Harshavardhana
c1e953b368 api: Set content-encoding properly if set. (#2245)
Additionally don't set content-type if not present, golang http
server automaticaly handles this and sets it automatically.
2016-07-20 12:40:20 -07:00
Krishna Srinivas
18728a0b59 XL/erasure-read: refactor erasure read and add tests (#2232) 2016-07-20 01:30:30 -07:00
Harshavardhana
cef26fd6ea XL: Refactor usage of reduceErrs and consistent behavior. (#2240)
This refactor is also needed in lieu of our quorum
requirement change for the newly understood logic behind
klauspost/reedsolom implementation.
2016-07-19 19:24:32 -07:00
Dee Koder
f67c930731 doc: Fixed spacing with respect to code blocks. (#2241) 2016-07-19 19:08:43 -07:00
GarimaKapoor
3589a58179 Update Minio-erasure-code-quickStart-guide.md 2016-07-19 17:59:05 -07:00
Dee Koder
e8155abc18 screenshot: Use the full path to the screenshot when embedding images (#2239) 2016-07-19 17:48:18 -07:00
Dee Koder
2e8360120d headings: We need to add a consistent heading for all docs. Adding Minio FreeBSD QuickStart Guide in the title. (#2233) 2016-07-19 14:56:34 -07:00
Dee Koder
02b191222c headings: Added standardized heading for this document. (#2234) 2016-07-19 14:56:20 -07:00
Dee Koder
b699795901 docs: Remove additional headings. Added standard heading. Include numbering. (#2235) 2016-07-19 14:30:32 -07:00
Harshavardhana
86d31e99d5 api: use checkAuth now at PutBucket, DeleteBucket handlers. (#2225)
Additionally add a unit test for isReqAuthenticated function.
2016-07-18 23:56:27 -07:00
Krishna Srinivas
897d78d113 erasureReadFile and erasureCreateFile testcases. (#2229)
* unit-tests: Unit tests for erasureCreateFile and erasureReadFile.

* appendFile() should return errXLWriteQuorum.

* TestErasureReadFileOffsetLength() tests erasureReadFile() for different offset and lengths.

* Fix for the failure seen in the erasure read unit test case. Issue #2227

* Move common erasure setup code to newErasureTestSetup()

* Review fixes. Add few more test cases for erasureReadFile.
2016-07-18 23:56:16 -07:00
Harshavardhana
1f706e067d api: xmlDecoder should honor contentLength. (#2226)
This is needed so that we avoid reading large amounts
of data from compromised clients.
2016-07-18 21:20:17 -07:00
Krishnan Parthasarathi
5cc9e4e214 fs/XL: Return IncompleteBody{} error for short writes (#2228) 2016-07-18 19:06:48 -07:00
Krishna Srinivas
27a5b61f40 tree-walk: optimize tree walk such that leaf detection of entries is delayed till the entry is sent on the treeWalkResult channel. (#2220) 2016-07-17 15:16:52 -07:00
Harshavardhana
aeac902747 API: ListBuckets doesn't have a body, we should never read the body. (#2218)
ListBuckets was incorrectly reading the body of the request, fix it.
2016-07-17 13:23:15 -07:00
Harshavardhana
aaf7803831 api: Requests should be differentiated if possible based on http router. (#2219)
In current master ListObjectsV2 was merged into ListObjectsHandler
which also implements V1 API as well.

Move the detection of ListObject types to its rightful place
in http router.
2016-07-17 12:32:05 -07:00
Krishna Srinivas
8cc163e51a Refactor xl.GetObject and erasureReadFile. (#2211)
* XL: Refactor xl.GetObject and erasureReadFile. erasureReadFile() responsible for just erasure coding, it takes ordered disks and checkSum slice.

* move getOrderedPartsMetadata and getOrderedDisks to xl-v1-utils.go

* Review fixes.
2016-07-16 08:35:30 -07:00
Harshavardhana
2d38046a5a utils: BucketNames with double periods and ip address should be rejected. (#2213)
Fixes #2212
2016-07-15 17:30:37 -07:00
Harshavardhana
cbb6b48b94 doc: update README.md 2016-07-15 16:12:54 -07:00
Harshavardhana
d0636d633d doc: Move FreeBSD.md to docs. 2016-07-15 16:09:01 -07:00
Harshavardhana
41187fc2ef docker: Fix docker edge build 2016-07-15 15:10:38 -07:00
koolhead17
204ec2c6c0 doc:README.md/Updated to sync with docs.minio.io (#2210)
* doc:README.md/Updated to sync with docs.minio.io

* doc:README.me/Modified the minio server output terminal to reflect new release changes.

* docs:README.md/Modified and changed location of other markdown files.
2016-07-15 15:03:59 -07:00
Krishnan Parthasarathi
3bce5db6d1 tests: Add tests to treeWalk for sortedness, recursive listing and isEnd behaviour (#2209) 2016-07-14 18:37:43 -07:00
Harshavardhana
35d438e0ff vendorize: update all vendorized packages. (#2206)
Bring in new changes from upstream for all the packages.

Important ones include
   - gorilla/mux
   - logrus
   - jwt
2016-07-14 14:59:20 -07:00
Krishna Srinivas
b090c7112e Refactor of xl.PutObjectPart and erasureCreateFile. (#2193)
* XL: Refactor of xl.PutObjectPart and erasureCreateFile.

* GetCheckSum and AddCheckSum methods for xlMetaV1

* Simple unit test case for erasureCreateFile()
2016-07-14 14:59:01 -07:00
Harshavardhana
af6109f89a update: Remove extraneous '/' in update message. (#2207) 2016-07-14 14:08:16 -07:00
Anis Elleuch
3f27734c22 Use normal color instead of forced white for users who have bright terminal background (#2200) 2016-07-13 14:27:36 -07:00
Harshavardhana
cdf1373f8e XL: Ignore and continue for cases when bucket does not exist. (#2205)
Fixes #2201
Fixes #2204
2016-07-13 13:44:33 -07:00
Krishnan Parthasarathi
45240f158d xl: Make namespace locking granular for PutObject (#2199) 2016-07-13 11:56:25 -07:00
Harshavardhana
0bd6b67ca5 server: Sort ips based on their last octet value. (#2198) 2016-07-13 06:34:59 -07:00
Harshavardhana
8c84df5e74 server: Change color codes for headings and sub-headings. (#2197)
This patch changes the color coding used for headings, sub-headings
and values as finalized.
2016-07-13 00:56:00 -07:00
Harshavardhana
dc3bafb194 XL: isQuorum rename as isDiskQuorum, word it properly. (#2196) 2016-07-13 00:29:48 -07:00
Harshavardhana
3b69b4ada4 server: Change server startup message. (#2195)
This change brings in the new agreed startup message
for the server.

Adds additional links point to Minio SDKs as well.
2016-07-12 23:21:18 -07:00
Krishnan Parthasarathi
0610527868 XL: PutObjectPart update checksum, re-read from xl.json for the part being written. (#2191) 2016-07-12 18:23:40 -07:00
Harshavardhana
0fcfb5df3b XL/fs: Change minioMetaBucket different than '.minio' config dir. (#2190)
This fixes corruption of config directory seen when minio server
exports 'home' directory.

```
minio server ~
```
2016-07-12 15:21:29 -07:00
Harshavardhana
623e0f9243 XL: listOnlineDisks should use modTime instead of version. (#2166)
This change is needed to make reading from objects future proof
in-terms of handling online disks. Our current counter is not
based on affirmative knowledge and relies on arithmetic sequence
which can lead to bugs.

Using modTime simplifies the understanding of `xl.json` and future
tooling / debugging of the format.
2016-07-12 15:20:31 -07:00
utsl42
e5cd35aad0 XL: GetObjectInfo() store and retrieve user-defined object metadata. (#2189) 2016-07-12 12:45:17 -07:00
Anis Elleuch
5cd629adca XL/fs: DeleteVol should not return error cleaning multipart dir for errVolumeNotFound (#2188) 2016-07-12 10:07:32 -07:00
Anis Elleuch
0fddf3fe17 Avoid creating tmp directories under .minio/tmp/ to facilitate cleaning (#2187) 2016-07-12 09:38:45 -07:00
karthic rao
ac6ff67546 Tool for running benchmark comparison of 2 commits (#2161) 2016-07-12 02:08:38 -07:00
Harshavardhana
126865e8df XL/bucket: Remove bucket should cleanup incomplete uploads as well. (#2173)
This behavior is in accordance with S3.

Fixes #2170
2016-07-12 01:01:47 -07:00
Krishnan Parthasarathi
1c82b81408 Rename parts/objects only on onlineDisks (#2185) 2016-07-11 22:53:54 -07:00
Bala FA
749a94f6c9 tests: Add tests for signature-jwt code (#2169)
Fixes #1989
2016-07-11 21:57:40 -07:00
Harshavardhana
e9647b5f12 API/CopyObject: Refactor the code and handle if-modified-since as well. (#2183)
This completes the S3 spec behavior for CopyObject API as reported
by `s3verify`.
2016-07-11 19:24:34 -07:00
Krishnan Parthasarathi
bef72f26db xl: Make locking more granular for PutObjectPart requests (#2168) 2016-07-11 17:24:49 -07:00
Harshavardhana
ede4dd0f9c server: update command should check for 3s from 1ms. (#2175)
Programmer error :-)
2016-07-11 16:22:10 -07:00
Bala FA
bfc59b7d50 tests: improve unit tests for xl-v1-metadata. (#2172)
Fixes #2124
2016-07-11 11:42:01 -07:00
Remco Verhoef
a162198623 implemented systemd script (#2167) 2016-07-11 03:55:40 -07:00
Harshavardhana
de468f92ec posix: ReadAll should handle the case when parent is not a dir. (#2163)
It can happen so that a read request can come for a file which
already has a parent i.e a file.

This fix handles this scenario - fixes #2047
2016-07-11 00:15:37 -07:00
Harshavardhana
d676e660c9 API/CopyObject: If-None-Match should return Precondition failed. (#2164) 2016-07-10 17:32:59 -07:00
Krishna Srinivas
aa7079fc7b XL/GetObject: If quorum not available during GetObject appropriate error should be returned. (#2135) 2016-07-10 17:12:22 -07:00
Harshavardhana
bdff0848ed server: Implement --ignore-disks for ignoring disks from healing. (#2158)
By default server heals/creates missing directories and re-populates
`format.json`, in some scenarios when disk is down for maintainenance
it would be beneficial for users to ignore such disks rather than
mistakenly using `root` partition.

Fixes #2128
2016-07-10 14:38:15 -07:00
Bala FA
0793237d94 tests: Move signature calculation in separate function. (#2160)
Previously newTestRequest() creates request object and returns
signature v4 signed request.  In TestCopyObject(), its required to add
headers later to the request and sign the request.

This patch introduces two new functions
* signRequest(): signs request using given access/secret keys.
* newTestSignedRequest(): returns new request object signed with given
  access/secret keys.

Fixes #2097
2016-07-10 11:10:59 -07:00
Bala FA
2a95eabb8a benchmarks: add parallel benchmarks for PutObject/GetObject. (#2159)
Fixes #2092
2016-07-10 11:08:45 -07:00
Krishnan Parthasarathi
bc8720406d Added specific error for InvalidObjectName (#2157) 2016-07-09 17:11:08 -07:00
Krishna Srinivas
ae80f8ca35 ObjectLayer/GetObject: Should return the right error value. Fix done in FS and XL. (#2133)
fixes #2117
2016-07-09 13:01:32 -07:00
Harshavardhana
5102a5877e API/handler: CopyObject make it behave in accordance with S3 spec. (#2155)
Fixes bugs found while running s3verify tool - fixes #2152
2016-07-09 12:13:40 -07:00
karthic rao
3341fe9b28 organizing the benchmarks in the right test files (#2154) 2016-07-09 00:45:49 -07:00
Harshavardhana
c0c8a8430e XL/PutObject: Add single putObject and multipart caching. (#2115)
- Additionally adds test cases as well for object cache.
- Adds auto-expiry with expiration and cleanup time interval.

Fixes #2080
Fixes #2091
2016-07-08 20:34:27 -07:00
karthic rao
b0c180b77c Test for ObjectLayer.GetObject() (#2153) 2016-07-08 18:26:04 -07:00
karthic rao
778b870b77 placing the http range error in objct-api-errors. (#2150) 2016-07-08 17:22:55 -07:00
Harshavardhana
cb415ef12e Merge pull request #2149 from harshavardhana/hash-order
XL/metadata: use new hashOrder algorithm for newXLMeta. (#2147)
2016-07-08 15:57:16 -07:00
Harshavardhana
6266328a85 XL/metadata: use new hashOrder algorithm for newXLMeta. (#2147) 2016-07-08 15:39:21 -07:00
frankw
63b3f1dcfd Use new algorithm to get fixed random order of disks (#2147) 2016-07-08 15:38:47 -07:00
Anis Elleuch
5ff1203fc0 Add PutObjectPart benchmark (#2145) 2016-07-08 14:28:06 -07:00
Harshavardhana
c3ab8bbd51 Turning off OSX builds for now. 2016-07-08 11:22:25 -07:00
Harshavardhana
a4a55bf134 tests: Fix erasure-readfile test formatting. 2016-07-08 11:05:30 -07:00
Harshavardhana
ec35330ebb XL/fs: GetObject should validate all its inputs. (#2142)
Fixes #2141
Fixes #2139
2016-07-08 07:46:49 -07:00
Harshavardhana
ca1b1921c4 XL: Implement ignore errors. (#2136)
Each metadata ops have a list of errors which can be
ignored, this is essentially needed when

  - disks are not found
  - disks are found but cannot be accessed (permission denied)
  - disks are there but fresh disks were added

This is needed since we don't have healing code in place where
it would have healed the fresh disks added.

Fixes #2072
2016-07-07 22:10:27 -07:00
Harshavardhana
4c21d6d09d tests: Remove racey failedDisks behavior in RenameObject tests. (#2138)
Additionally also initialize namespace lock only once per test
run, there is no reason to initialize it multiple times to avoid
races.

Fixes #2137
2016-07-07 19:50:44 -07:00
Bala FA
5ec7734d88 FS: Check offset is within object size in GetObject() (#2123)
Fixes #2118
2016-07-07 19:49:45 -07:00
karthic rao
2c837128ef Object layer tests revamp for individual execution (#2134) 2016-07-07 15:05:51 -07:00
Bala FA
282044d454 http: handle possible range requests properly. (#2122)
Previously range string is not validated against various combination
of values.  This patch fixes the issue.

Fixes #2098
2016-07-07 15:05:18 -07:00
Harshavardhana
8e53064bb4 XL/format: Initialize missing meta volume. (#2131)
Fixes #2127
2016-07-07 15:01:54 -07:00
Krishna Srinivas
f55093cdd6 multipart: During multipart list the listing go-routine should be saved to the List-pool. (#2130) 2016-07-07 09:06:35 -07:00
Harshavardhana
ddf3245677 xl/fs: offset and length cannot be negative. (#2121)
Fixes #2119
2016-07-07 01:30:34 -07:00
Harshavardhana
169c72cdab vendor: Bring new updates from blake2b-simd repo. (#2094)
This vendorization is needed to bring in new improvements
and support for AVX2 and SSE.

Fixes #2081
2016-07-06 18:24:31 -07:00
Nick Craig-Wood
8c767218a4 URL Encode X-Amz-Copy-Source as per the spec (#2114)
The documents for COPY state that the X-Amz-Copy-Source must be URL encoded.

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html
2016-07-06 15:42:17 -07:00
Bala FA
a51bb1d728 http: handle request range like Amazon S3. (#2112)
Fixes #2098
2016-07-06 12:50:24 -07:00
Krishna Srinivas
26b7d79a85 XL/ObjectCache: DeleteObject() should delete the object from the object cache. (#2113)
fixes #2103
2016-07-06 10:25:42 -07:00
Krishna Srinivas
01cbacd803 object-cache: use golang bytes.Buffer and bytes.NewReader instead of custom implementation. (#2108) 2016-07-06 01:29:49 -07:00
Harshavardhana
7bde27032d signv4: Validate preSigned payload properly. (#2106)
We need to only validate presigned payload only
if the payload is requested for, with default payload
i.e 'UNSIGNED-PAYLOAD' we don't need to validate.

Fixes #2105
2016-07-05 21:00:20 -07:00
Bala FA
44ae7a037b fix: allocate buffer to required size than readSizeV1 (#2095)
Refer #2077
2016-07-05 20:59:54 -07:00
karthic rao
a35341448f XL: Cache: Purging partially cached content upon erasureReadFile failure (#2104) 2016-07-05 12:48:54 -07:00
Krishna Srinivas
d6dfcd0ba7 unit-tests: Unit tests for functions in erasure-readfile.go (#2090) 2016-07-05 11:41:25 -07:00
Harshavardhana
8ddf52021a server: Bring in s3 compatibility fixes. (#2099)
This patch fixes majority of discrepant messages and responses
previously reported.

There are few discrepancies observed

- S3 is not honoring 'If-Modified-Since' header.
- We do not implement object policy, S3 returns a different response in this category.
- Adding new headers causes signature mismatch, but Minio server is fine for example
  TestCopyObject() to be fixed by moving the signature logic out.
  Relevant bug - https://github.com/minio/minio/issues/2097

Fixes #1955
2016-07-05 01:06:30 -07:00
Harshavardhana
8a028a9efb handler/PUT: Handle signature verification through a custom reader. (#2066)
Change brings in a new signVerifyReader which provides a io.Reader
compatible reader, additionally implements Verify() function.

Verify() function validates the signature present in the incoming
request. This approach is choosen to avoid complexities involved
in using io.Pipe().

Thanks to Krishna for his inputs on this.

Fixes #2058
Fixes #2054
Fixes #2087
2016-07-05 01:04:50 -07:00
Bala FA
0540863663 fix: use readSizeV1 wherever applicable. (#2093) 2016-07-04 19:21:15 -07:00
Harshavardhana
4cfbdb1bf0 server: Remove deadcode/deprecated code. (#2088) 2016-07-04 14:46:38 -07:00
Krishna Srinivas
1ec86dac2c server-tests: unify XL and FS tests into common code. server_test.go contains common test code. server_xl_test.go contains XL tests specific to XL. (#2089) 2016-07-04 13:18:41 -07:00
Krishna Srinivas
7a8b8cd0a1 tree-walk: unify FS and XL tree-walk with functional approach. (#2027) 2016-07-04 01:49:27 -07:00
karthic rao
a8a3e95835 Api/Bucket-Policy: Handler tests (#2074) 2016-07-03 22:35:30 -07:00
Bala FA
1ad5fb8f76 posix: checkDiskFree() also checks free inodes. (#2086)
Previously checkDiskFree() checks for free available space.  This
patch enables checkDiskFree() also checks for free inodes in linux and
free clusters in windows.

Fixes #2075
2016-07-03 22:34:45 -07:00
Bala FA
52b55afce0 FS: check whether disk format is FS or not. (#2083)
Fixes #2060
2016-07-03 20:01:40 -07:00
Harshavardhana
355f06cfea tests: Add urlEncode tests. (#2078) 2016-07-03 19:25:04 -07:00
Harshavardhana
d2557bb538 XL: GetObject caching implemented for XL. (#2017)
The object cache implementation is XL cache, which defaults
to 8GB worth of read cache. Currently GetObject() transparently
writes to this cache upon first client read and then subsequently
serves reads from the same cache.

Currently expiration is not implemented.
2016-07-03 17:15:38 -07:00
Bala FA
8d4365d23c tests: add unit test for posix functions. (#2037)
Unit tests for posix operations.

* MakeVol
* DeleteVol
* StatVol
* ListVols
* DeleteFile
* AppendFile
* RenameFile
* StatFile

Fixes #2021
2016-07-03 11:17:08 -07:00
karthic rao
55ae7cac42 api/bucket-policy: Refactor, Handler test. (#2071)
PR contains,
- New setup utilities for running object handler tests. Here is why they are essential, 
    - Unit tests have to be run in isolation without being have to be associated with other functionalities which are not under test. 
    - The integration tests follows the philosophy of running a Test Server and registers all handlers and fires HTTP requests over the socket to simulate the system functionality under usual work load scenarios and test for correctness. But this philosophy cannot be adopted for running unit tests for HTTP handlers. 
    - Running Unit tests for API handlers, 
        - Shouldn't run a test server. Should purely call the handlers `ServeHTTP` under isolation. 
        - Shouldn't register all handlers, should only register handlers under test and so that the system is close to be in an isolated setup. 

- As an example PutBucketPolicy test is illustrated using the new setup. Exhaustive cases has to be added and has been listen in TODO for now.
2016-07-02 19:05:16 -07:00
Harshavardhana
48ac34919f browser: Add new release for ui-assets.go (#2070)
update `ui-assets.go` using `x-amz-date` for JSON rpc.
2016-07-02 10:54:17 -07:00
Harshavardhana
d64c3fd464 posix: Return errDiskNotWritable during disk initialization. (#2048)
It can happen that minio server might not have
writable permissions on the export paths command line.

Fixes #2035
2016-07-02 01:59:28 -07:00
Harshavardhana
e5dd917c37 handlers/generic: Remove support for 'x-minio-date' (#2064) 2016-07-02 01:51:17 -07:00
Krishna Srinivas
eb5f782c74 object-handler: skip sha256 calculation if x-amz-content-sha256=="UNSIGNED-PAYLOAD" (#2038)
fixes #2024 #2056
2016-07-01 14:34:40 -07:00
Harshavardhana
734e779b19 XL/erasureCreate: Create a limit reader if size is specified. (#2059)
This is needed so that we only write data which was requested
for, using a limit reader avoids spurious reads on the incoming
client data. Additionally using limit reader provides server
safety from rogue clients sending copious amounts of data (for
example a denial of service attack).

This patch also caters for size == -1 when content encoding from
a client is set as chunked, we happily read till io.EOF
2016-07-01 14:33:28 -07:00
Krishna Srinivas
3f2b4d9dc2 Show "https" in the "minio server export/" output if certificates are available. (#2065)
fixes #2036
2016-07-01 14:32:53 -07:00
Bala FA
cd1c2db864 posix-utils: fix path handling in windows. (#2053) 2016-07-01 11:44:07 -07:00
karthic rao
48aa5f2199 api/bucket-policy: Add unit tests for more coverage, fixes couple of bugs. (#2055)
Changes to ResourceMatch logic.
Test for action match function.
2016-06-30 23:49:59 -07:00
Krishnan Parthasarathi
bcb822c390 Send XML header before the first of whitespace chars (#2046)
* Sent XML header before the first of whitespace chars

XML parsing fails in aws cli due to unexpected whitespace character. To
fix this, we send the xml header before we send the first whitespace
character, if any.

* Fix race between sendWhiteSpaceChars and completeMultiUploadpart
2016-06-30 18:48:50 -07:00
Krishnan Parthasarathi
285a94d2c0 Ignored errors in cleanup in commitXLMetadata (#2044)
Deletion of tmp files where xl metadata was saved before the commit
operation doesn't change the error returned to the caller. So, it is to
be ignored.
2016-06-30 16:37:51 -07:00
Krishnan Parthasarathi
64899e5197 xl: Used unique tmp file to update xl.json in putObjectPart (#2043)
An in-place update to xl.json amidst concurrent PutObjectPart operations
lead to racy updates to xl.json making it un-parseable. To avoid this,
we create a unique tmp file where updates to xl.json are staged before
renaming it to the final location.
2016-06-30 16:28:01 -07:00
Danilo Pereira
812554087f getCertsPath should use getConfigPath instead of defaulting to users homedir. (#2039)
Fixes #2028
2016-06-30 15:49:18 -07:00
Bala FA
57bc08cc7e posix: remove disk free space check for read-only and delete methods. (#2033) 2016-06-29 11:25:35 -07:00
Harshavardhana
0e3907072c XL/fs: Initialize export paths supplied on command line (#2020)
Fixes #2013
2016-06-29 03:13:44 -07:00
karthic rao
8e8f6f90a4 adding detailed comments to server_test (#2032) 2016-06-29 02:30:36 -07:00
Bala FA
4c1a11aae6 XL: allow meta bucket name appended with tmp meta prefix. (#2007) 2016-06-29 02:28:46 -07:00
Harshavardhana
ae936a0147 XL: Relax write quorum further to N/2 + 1. (#2018)
This changes behavior in some parts of the code
as well address it.

Fixes #2016
2016-06-29 02:10:40 -07:00
Harshavardhana
d484157d67 XL/bitrot: Migrate to new blake2b-simd SIMD optimized implementation. (#2031)
Thanks for Frank Wessels <fwessels@xs4all.nl> for all the heavy lifting work.

Comparative benchmarks are as below.
```
benchmark                old ns/op     new ns/op     delta
BenchmarkHash64-4        742           411           -44.61%
BenchmarkHash128-4       681           346           -49.19%
BenchmarkWrite1K-4       4239          1497          -64.69%
BenchmarkWrite8K-4       33633         11514         -65.77%
BenchmarkWrite32K-4      134091        45947         -65.73%
BenchmarkWrite128K-4     537976        183643        -65.86%

benchmark                old MB/s     new MB/s     speedup
BenchmarkHash64-4        86.18        155.51       1.80x
BenchmarkHash128-4       187.96       369.10       1.96x
BenchmarkWrite1K-4       241.55       683.87       2.83x
BenchmarkWrite8K-4       3897.06      11383.41     2.92x
BenchmarkWrite32K-4      977.48       2852.63      2.92x
BenchmarkWrite128K-4     243.64       713.73       2.93x
```

Fixes #2030
2016-06-29 02:06:35 -07:00
Harshavardhana
796fe165c7 server: Minor command line doc change for XL. (#2022) 2016-06-29 02:05:57 -07:00
Harshavardhana
3ac39ff107 XL: Change minimum disks supported to 6 now. (#2023)
This change co-incides with another patch set which
reduces the writeQuorum requirement. With the
write quorum change it is now possible to support
6 disk configuration.
2016-06-29 02:05:29 -07:00
Krishnan Parthasarathi
b6b9e88e47 Added unit-tests for treeWalkPool (#1969)
* Added unit-tests for treeWalkPool
* Added unit tests for tree-walk-fs
* Added period at the end of all comments.
* FS/XL: Unified tree walk tests for both backends
* Added disk failure related tests for treewalk

Replaced removeRandomDisks with removeDiskN. There is no need to
randomize disks that fail while the distribution of chunks in XL during
erasure coding data is random.
2016-06-28 22:32:00 -07:00
karthic rao
59366d8f4c Benchmarks for ObjectLayer.PutObject() (#2029) 2016-06-28 22:12:36 -07:00
Harshavardhana
748dc80047 API: add writePartTooSmallErrorResponse to extend standard error responses. (#2005)
This function is added to extend the standard error responses.
Which is needed in some cases for example CompleteMultipartUpload
should respond with ErrPartTooSmall error when parts uploaded are
lesser than 5MB (i.e minimum allowed size per part).

Fixes #1536
2016-06-28 14:51:49 -07:00
karthic rao
6dcfa7b046 Fix for tests leaving out temp directories (#2025) 2016-06-28 04:21:52 -07:00
Krishnan Parthasarathi
a854e8cc5c api: Sent ErrPreconditionFailed on If-Match failure (#2009)
* api: Sent ErrPreconditionFailed on If-Match failure

ref:
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#ErrorCodeList

* tests: Added functional tests for GetObject w/ If-Match headers set

* tests: Used verifyError to simplify errorCode and description matching on error
2016-06-28 01:18:18 -07:00
karthic rao
76f6533f8d Adding detailed comments for server_xl_test. (#2011) 2016-06-27 23:54:56 -07:00
Harshavardhana
4db2b03312 XL: Rename objectN to part.N (#2019)
Fixes #2015
2016-06-27 21:42:33 -07:00
Krishna Srinivas
5291db60c6 XL/erasure: refactor erasureReadFile. Move parallelRead into a separate function. (#2008) 2016-06-27 13:24:55 -07:00
Harshavardhana
2e1f66c37d XL: Handle quorum situations properly for write operations. (#1986)
Adds two test cases one for

 - PutObject when write quorum is not available.
 - PutObjectPart when write quorum is not available.

Fixes #1951
2016-06-27 10:01:09 -07:00
Bala FA
c88720ea2c XL/listObjects: Ignore entry if getObjectInfo() returns errFileNotFound (#2004)
Fixes #1956
2016-06-26 22:10:22 -07:00
karthic rao
ce7d5eddbc Misspell warnings fix (#2001) 2016-06-26 22:05:48 -07:00
Harshavardhana
0d3a9c9438 XL: Add tests for checkSufficientDisks, storageInfo. (#1988) 2016-06-26 19:48:02 -07:00
Harshavardhana
293ba00249 posix: Re-do tests for readDir(). (#1996) 2016-06-26 19:31:53 -07:00
Krishnan Parthasarathi
d0be09fdd3 object: checkETag compares quoted ETags properly (#1997)
Previously, checkETag didn't handle ETags with leading and trailing
double quotes. e.g "abcdef1234" == "\"abcdef1234\"" would return false.
Now, checkETag function canonicalizes the ETags passed as arguments by
removing one leading/trailing double quote.
2016-06-26 18:10:08 -07:00
Remco Verhoef
1e52759c3c fix typo (#1987) 2016-06-26 11:27:04 -07:00
Harshavardhana
9add048f3c erasure: Add erasure encode/decode unit tests. (#1911)
Fixes #1819
2016-06-26 03:32:49 -07:00
Harshavardhana
936a916e78 server: Add connection rate limiter for server. (#1977) 2016-06-26 03:18:07 -07:00
Harshavardhana
57146fbbb8 main: minio --help should print regardless of root. (#1985)
Remove root check entirely.

Fixes #1964
2016-06-26 03:03:52 -07:00
karthic rao
3d02f7471e Benchmarks for various object sizes for FS/XL GetObject (#1984) 2016-06-25 20:22:04 -07:00
karthic rao
b2d9a46cbb Cleaning up/refactoring tests to be more extensible (#1970) 2016-06-25 19:07:44 -07:00
Harshavardhana
42286cba70 XL: Implement new ReadAll API for files which are read in single call. (#1974)
Add a unit test as well.
2016-06-25 14:51:06 -07:00
karthic rao
ed2fdd90b0 fs: Fix GetObject failure to read large blocks. (#1982)
Add relevant test cases as well for verifying this
part of the codebase.

Fixes #1979
2016-06-25 03:03:27 -07:00
Krishna Srinivas
cb1200a66d XL/erasure-read: read disks in parallel. (#1975)
On read failure, fallback to reads from other
disks also happen in parallel.
2016-06-24 18:02:10 -07:00
Krishnan Parthasarathi
a3a310cde8 Moved tree-walk-fs to use tree-walk-pool (#1978) 2016-06-24 16:41:57 -07:00
Bala FA
f625392211 tests: add unit test for posix.readDir() (#1967)
Fixes #1820
2016-06-24 14:32:08 -07:00
Harshavardhana
e8990e42c2 XL: Make allocations simpler avoid redundant allocs. (#1961)
- Reduce 10MiB buffers for loopy calls to use 128KiB.
- start using 128KiB buffer where needed.
2016-06-24 02:06:23 -07:00
Harshavardhana
ff9fc22c72 posix: Mkdir() and OpenFile() should honor umask. (#1972)
Adds two unit tests for validation as well.

Fixes #1965
2016-06-23 20:19:27 -07:00
Harshavardhana
41c089a7e0 XL: Add mis-aligned GetObject() test. (#1960) 2016-06-22 21:42:24 -07:00
Harshavardhana
5725f3c809 Merge pull request #1958 from krisis/unittest/unc-path
Added unit tests for UNC path handling in windows
2016-06-22 17:19:40 -07:00
Harshavardhana
f4830162a4 XL: Format heal should re-allocate new UUIDs not reuse. (#1953)
This patch also supports writing to a temporary file and renaming
rather than appending to an existing file. This helps in avoiding
inconsistent files.
2016-06-22 17:18:31 -07:00
Harshavardhana
e10934a88e bitrot: Start using blake2b algorithm and remove sha512 usage. (#1957)
Fixes #1952
2016-06-22 17:13:26 -07:00
Krishnan Parthasarathi
a07751f61b Added tests to validate disk name length near MAX_PATH 2016-06-22 15:30:41 -07:00
Krishnan Parthasarathi
0766e903e3 Added unit tests for UNC path handling in windows 2016-06-22 15:30:41 -07:00
Harshavardhana
75dddfb2ae Merge pull request #1959 from krishnasrinivas/parallel-reads
Parallel reads in erasure-read
2016-06-22 15:05:35 -07:00
Harshavardhana
9b82e64a11 XL/erasure-read: Avoid memory copy, write to writer directly all the dataBlocks. 2016-06-23 02:06:57 +05:30
Harshavardhana
bdf8738076 lock: Add unit tests for namespace lock (#1922)
Fixes #1821
2016-06-22 12:27:47 -07:00
Krishna Srinivas
d4bea5fbf8 XL/erasure-read: Add Comments and enable bitrot detection. 2016-06-23 00:34:47 +05:30
Krishna Srinivas
17efaaa902 XL/erasure-read: Support parallel reads from disks. 2016-06-23 00:34:47 +05:30
Krishnan Parthasarathi
78ae696749 Added couple of unit-tests to xlObjects (#1950)
* Added couple of units to xlObjects

* Used test_utils for initialize/destroying xlObjects
2016-06-21 15:48:27 -07:00
Harshavardhana
3fa95f5263 docker: Remove unneeded docker files and makefile tags. 2016-06-21 15:31:30 -07:00
karthic rao
ba5bb4a127 TestServer introduction and revamp of functional tests. (#1940)
Allows for easy creation of Test server with temp backend.

changes
2016-06-21 12:10:18 -07:00
Harshavardhana
409b4ddecb api: MakeBucket should set proper bucket location. (#1948)
Fixes #1942
2016-06-20 23:25:18 -07:00
Harshavardhana
ad779a8ba4 XL: Enable tests for large GetObject. (#1947)
Ref #1946
2016-06-20 22:08:36 -07:00
Bala FA
7d757033f2 erasure-readfile: Use chunk size to read from each disk for a block. (#1949)
A block of data is split into data chunk and each data chunk is
written to each disk.  Previously block size was used to read data
chunk which returns corrupted data.

This patch fixes the issue by reading chunk sized data from each disk
and assembles a block.

Fixes #1939
2016-06-20 21:40:10 -07:00
Krishnan Parthasarathi
393c504de0 Renaming a part from tmp namespace needs to be handled different from… (#1944)
* Renaming a part from tmp namespace needs to be handled different from renaming of an object

* Made argument passing in xl.rename and xl.undoRename explicit
2016-06-20 19:11:55 -07:00
Krishnan Parthasarathi
6143c87c3a Make ioErrCount updates go-routine safe (#1943)
* Make ioErrCount updates go-routine safe

* Made ioErrCount int32 instead of *int32

... and implemented StorageAPI on *posix as opposed to posix type.
This is consistent with the thumb-rule that if a value of a type is
modified as part of the interface implementation then we implement the
interface on pointer to that type.
2016-06-20 16:57:14 -07:00
Bala FA
8559e89494 XL: fix getBlockInfo() to return correct end block (#1941)
If requested offset/length of an object is equal to
erasureInfo.BlockSize, getBlockInfo() returns one more block added to
actual end block.  This patch fixes the issue.

This patch also adds unit test for get objects with big files.
2016-06-20 14:23:25 -07:00
Krishna Srinivas
468ca4ccda XL/Unittest: Add testcase for xlMetaV1{} and its methods. (#1938)
fixes #1822
2016-06-20 07:35:41 -07:00
Aakash Muttineni
4ee2136b28 Unit tests for PUT object when object already exists (#1904)
* fs/xl tests for multiple put object requests
* xl fix for put object on directory
* Unit tests fix windows test issue.
2016-06-20 06:18:47 -07:00
Yurii Rashkovskii
80d83220ad INSTALLGO.md mentions Go 1.5+ for OS X (#1936)
However, current requirement is 1.6, so the file has been updated to reflect that.
2016-06-20 06:17:57 -07:00
Bala FA
fb10c09da7 posix-utils: remove unused isValidPath() (#1937) 2016-06-20 06:17:36 -07:00
Bala FA
2f136e92f7 posix: cleanup usage of fmt.Println() (#1934) 2016-06-19 18:52:19 -07:00
Harshavardhana
50d25ca94a XL: Change AppendFile() to return only error (#1932)
AppendFile ensures that it appends the entire buffer. Returns
an error otherwise, this patch removes the necessity for the
caller to look for 'n' return on short writes.

Ref #1893
2016-06-19 15:31:13 -07:00
Harshavardhana
e1aad066c6 XL: CompleteMultipart should ignore last part is 0bytes. (#1931)
Fixes #1917
2016-06-19 14:51:20 -07:00
Bala FA
1ea1dba528 erasure-readfile: write to given Writer than returning buffer. (#1910)
Fixes #1889
2016-06-19 13:35:26 -07:00
Krishna Srinivas
c41bf26712 Unit tests: add unit tests for listv1/v2 for list bucket handler. (#1933)
fixes #1818
2016-06-19 13:33:00 -07:00
Harshavardhana
8c0942bf0d XL: Remove usage of reduceErr and make it isQuorum verification. (#1909)
Fixes #1908
2016-06-18 00:27:51 +05:30
Harshavardhana
7f38f46e20 vendor: update klauspost/reedsomon package with upstream changes. (#1912) 2016-06-17 15:16:26 +05:30
Krishna Srinivas
466a2e01f1 XL/Erasure: Blocksize for object-part should be derived from what was decided during xl.NewMultipartUpload which creates xl.json. (#1920)
fixes #1919
2016-06-17 12:47:15 +05:30
Krishna Srinivas
d31b38aac8 XL/GetObject: pick the xl.json with highest version for metadata information. (#1914)
fixes #1913
2016-06-17 10:56:18 +05:30
Krishna Srinivas
365f80efa3 XL/DeleteObject: delete call on a prefix should not delete the entire tree structure. (#1916)
fixes #1915
2016-06-17 10:48:43 +05:30
Anand Babu (AB) Periasamy
f51d34cedd Do not guess content-type for objects with no extension (#1918) 2016-06-17 10:12:02 +05:30
Krishnan Parthasarathi
129ebbd685 object layer: Send 200 OK and whitespace chars (#1897) 2016-06-16 09:01:06 +05:30
Krishna Srinivas
e2743d05e8 FS: remove .minio directory if .minio/multipart is empty. (#1899)
fixes #1886
2016-06-16 08:50:38 +05:30
Krishna Srinivas
de1c7d33eb XL: appendFile should return error if quorum is not met. (#1898)
Fixes #1890
2016-06-15 00:24:49 +05:30
karthic rao
afc3102488 Adding format.json during FS initialization (#1896) 2016-06-14 14:09:40 +05:30
Harshavardhana
23c88ffb1d XL/format: Fix a bug in checkDisksConsistency. (#1894) 2016-06-14 01:12:15 -07:00
Harshavardhana
ed4fe689b4 posix: Support UNC paths on windows. (#1887)
This allows us to now use 32K paths names on windows.

Fixes #1620
2016-06-13 02:53:09 -07:00
Harshavardhana
4ab57f7d60 server: terminal width should fallback to 80x25. (#1895)
Some environments might disable access to `/dev/tty`, fall
back to '80' in such scenarios.

Move to 'cheggaaa/pb' package for better cross platform
support on fetching terminal width.

Fixes #1891
2016-06-12 19:35:28 -07:00
karthic rao
276282957e Test for Complete Multipart Upload. (#1888) 2016-06-10 18:43:16 +05:30
Harshavardhana
71632b375e docs: Add comments for each data types. (#1881) 2016-06-09 06:24:11 -07:00
Aakash Muttineni
6f3bd76754 api: Add new bucket policy nesting error (#1883)
* Added ErrPolicyNesting which is returned when nesting of policies has occured
* Replaces ErrMalformedPolicy in the case of nesting
* Changed test case in bucket-policy-parser_test.go (ErrMalformedPolicy -> ErrPolicyNesting)
2016-06-09 01:53:56 -07:00
Bala FA
f2765d98a8 XL: set write quorum (no. of disk / 2) + 2 (#1876)
Previously write quorum was set to (no. of disk / 2) + 3.  As per new
change, the write quorum is set to (no. of disk / 2) + 2.  This helps
to accommodate one more failure of disk.
2016-06-08 22:12:36 -07:00
Bala FA
61598ed02f posix: return errFaultyDisk on I/O errors. (#1885)
When I/O error is occured more than allowed limit, posix returns
errFaultyDisk.

Fixes #1884
2016-06-08 22:02:10 -07:00
Krishna Srinivas
1b9db9ee6c FS/PutObject: Read() data should be handled even in case of EOF. (#1864)
Fixes #1710
2016-06-08 22:00:31 -07:00
Harshavardhana
51f3d4e0ca XL/multipart: statPart should ignore errDiskNotFound. (#1862)
startPart should also take uploadId and partName as arguments.
2016-06-07 18:15:04 -07:00
Bala FA
d13e6e7156 XL: return error if DeleteObject() fails. (#1878)
Previously DeleteObject() does not return any error if write quorum is
not available.  This patch fixes the issue by returning errors.
2016-06-07 11:35:03 -07:00
Bala FA
d32f3288f8 XL: return false only if given prefix doesn't exist in all disks (#1877)
Previously xl.isObject() returns false if one of the disk doesn't have
the object.  Its possible that object may be present in another disk.

This patch fixes the issue by returning false only if given prefix
doesn't exist in all disks.

Fixes #1855
2016-06-07 11:02:12 -07:00
Harshavardhana
c5b6cb2420 Merge pull request #1867 from balamurugana/devel
cleanup: remove unused waitCloser.
2016-06-06 20:32:33 -07:00
Bala.FA
2eb6fa3fce cleanup: remove unused waitCloser. 2016-06-07 07:38:18 +05:30
Krishna Srinivas
acc393ba8b XL/tree-walk: Added comments, changed variable names and structure fields to improve code readability. (#1856) 2016-06-05 11:55:45 -07:00
Harshavardhana
37551a2ad3 Merge pull request #1857 from harshavardhana/erasure
erasure: Fix block index matching.
2016-06-05 09:35:31 -07:00
Harshavardhana
c6ac3fa6db erasure: Fix block index matching.
This patch fixes an important issue of block reconstruction upon block corruption.
2016-06-05 06:19:58 -07:00
Harshavardhana
18b3871705 Add erasure code. 2016-06-03 12:50:36 -07:00
Harshavardhana
73ddb5be75 Merge pull request #1850 from harshavardhana/list-rewrite
XL: Implement ListObjects channel and pool management.
2016-06-03 12:30:06 -07:00
Krishna Srinivas
002c5bf7dd XL: Treewalk handle all the race conditions and blocking channels. 2016-06-03 12:17:54 -07:00
Harshavardhana
1cf1532ca3 XL: Implement ListObjects channel and pool management. 2016-06-03 12:17:54 -07:00
Harshavardhana
70a1231f02 Merge pull request #1849 from harshavardhana/multipart
XL/PutObject: Handle all pending cases of DiskNotFound.
2016-06-03 11:58:04 -07:00
Harshavardhana
82fd907933 XL/PutObject: Handle all pending cases of DiskNotFound. 2016-06-03 11:40:44 -07:00
Harshavardhana
f39a6f8df7 Merge pull request #1852 from minio/harshavardhana-patch-1
Fix download link
2016-06-03 01:17:54 -07:00
Harshavardhana
f6013c46ea Fix download link 2016-06-03 00:05:32 -07:00
Harshavardhana
da069a18c4 Merge pull request #1847 from krisis/patch-1
Created ISSUE_TEMPLATE with basic information
2016-06-03 00:01:48 -07:00
Harshavardhana
5108ba6eb1 Merge pull request #1728 from minio/rewrite-xl
XL/FS: Rewrite in new format.
2016-06-02 23:19:17 -07:00
Krishnan Parthasarathi
1213bf9fa1 Created ISSUE_TEMPLATE with basic information
Added an issued template to gather as much useful information that we may need to resolve an issue while the user has access to the relevant details.
2016-06-03 10:31:08 +05:30
Krishna Srinivas
b00ac40c35 XL/PutObject: Calculate size if not provided by the client and update xl.json with the correct size. (#1844) 2016-06-02 17:09:47 -07:00
Harshavardhana
fb95c1fad3 XL: Bring in some modularity into format verification and healing. (#1832) 2016-06-02 16:34:15 -07:00
Krishna Srinivas
aa1d769b1e FS/Multipart: remove uploads.json on complete-multipart if no more uploadIDs are present for the object. (#1843)
Fixes #1835
2016-06-02 15:54:00 -07:00
Krishna Srinivas
611c892f8f FS/Multipart: Lock() to avoid race during PutObjectPart. (#1842)
Fixes #1839
2016-06-02 15:19:13 -07:00
Harshavardhana
67bba270a0 FS: Cleanup and Fix all multipart related operations. (#1836) 2016-06-02 12:18:56 -07:00
Harshavardhana
de21126f7e XL: Re-align the code again. 2016-06-02 01:54:06 -07:00
Harshavardhana
ae311aa53b XL: Cleanup, comments and all the updated functions. (#1830) 2016-06-01 16:43:31 -07:00
Krishna Srinivas
9b79760dcf XL/heal: heal missing format.json on replaced drives. (#1828)
fixes #1817
2016-06-01 16:15:56 -07:00
Bala FA
116b5607d7 server: fix to have readable timeout value (#1823) 2016-06-01 09:14:50 -07:00
Krishna Srinivas
614c770b5d List Objects version 2. (#1815)
object: List Objects v2 support
2016-05-31 22:10:55 -07:00
Harshavardhana
c493ab5d0d XL: Bring in sha512 checksum support. (#1797) 2016-05-31 20:23:31 -07:00
Bala FA
db2fdbf38d erasure: allocate buffer only for non-nil disk (#1811) 2016-05-31 11:55:50 -07:00
Krishna Srinivas
89f65333fb XL/Multipart: Introduce "deleted" field for uploads.json (#1810)
To future proof backend in case #1805 becomes an issue.
2016-05-31 11:54:01 -07:00
Krishna Srinivas
22511dc4c7 XL/Multipart: During list-multipart-uploads ignore errFileNotFound and errDiskNotFound errors. (#1813)
Fixes #1795
2016-05-31 11:53:28 -07:00
karthic rao
1947ae198e Adding read nad write timeout for unresponsive client connectinos (#1809) 2016-05-31 11:53:21 -07:00
Harshavardhana
2e4ab71303 Web: Update with ui changes. (#1808) 2016-05-31 02:01:02 -07:00
Harshavardhana
445dc22118 XL: Cleanup and add more comments. (#1807) 2016-05-30 16:51:59 -07:00
karthic rao
ffc2b3c304 Test for ListObjectParts. (#1802) 2016-05-30 14:36:33 -07:00
Krishnan Parthasarathi
967c2b2940 Handled possible short writes to httpResponseWriter (#1804)
* XL: Handled possible short writes to httpResponseWriter

* Added tests for Range Header combinations
2016-05-30 11:27:15 -07:00
Krishna Srinivas
b466f27705 Nslock fixes (#1803)
* XL/Multipart: Support parallel upload of parts by doing NS locking appropriately.

* XL/Multipart: hold lock on the multipart upload while aborting.
2016-05-30 11:26:10 -07:00
Harshavardhana
a4a0ea605b XL: Fix GetObject erasure decode issues. (#1793) 2016-05-29 15:38:14 -07:00
Harshavardhana
5e8de786b3 XL: Truly use unique id's in temp directory. (#1790)
This also helps in avoiding cleaning up directories after.

Additionally this patch also fixes the problem of Range offsets.
2016-05-29 00:42:09 -07:00
Harshavardhana
feb337098d XL: bring in new storage API. (#1780)
Fixes #1771
2016-05-28 16:12:51 -07:00
Krishnan Parthasarathi
c87f259820 Remove parts that are missing in CompleteMultipartUpload (#1786)
* Remove parts that are missing in CompleteMultipartUpload

* Moved isUploadIDExists under proper namespace locks

* Moved code that deletes part files to a function
2016-05-28 15:15:53 -07:00
karthic rao
7278b90fe1 Adding defer to the lock (#1785) 2016-05-28 15:15:53 -07:00
Krishna Srinivas
41a5b3908b XL/ListParts: take the size from xl.json instead of backend file size as it will be different. (#1781)
Fixes #1779
2016-05-28 15:15:53 -07:00
Krishna Srinivas
3fb0b5e455 XL/Multipart: check existance upload uploadID after lock. (#1778)
Fixes #1767
2016-05-28 15:15:53 -07:00
Harshavardhana
ba8bdec077 XL: ListObjects should not list when delimiter and prefix are '/'. (#1777) 2016-05-28 15:15:53 -07:00
Harshavardhana
27cc8a6529 erasure: read only dataBlocks if we have enough. (#1776)
Reconstruct with parity blocks if we don't have enough data blocks.
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
302ec27fa2 Fixed race during parallel PutObjectPart requests (#1775)
The race is between two parallel PutObjectPart requests updating partsInfo in xl.json.
Previously, it was being updated under a RLock().
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
5f679d9d1e Rename back multipart objects if read/write Quorum was unavailable (#1773) 2016-05-28 15:15:53 -07:00
Bala FA
51bb613fdf pkg/safe: remove temporary file on failure (#1774) 2016-05-28 15:15:53 -07:00
Harshavardhana
d65101a8c8 XL: Implement strided erasure distribution. (#1772)
Strided erasure distribution uses a new randomized
block distribution for each Put operation. This
information is captured inside `xl.json` for subsequent
Get operations.
2016-05-28 15:15:53 -07:00
Krishna Srinivas
6dc8323684 FS/ListMultipart: Fix FS list-multipart to work for unit test cases. 2016-05-28 15:15:53 -07:00
Krishna Srinivas
616a257bfa XL/Multipart: isMultipartUpload() checks for presence of uploads.json on a random disk. 2016-05-28 15:15:53 -07:00
Krishna Srinivas
3487b3c095 Multipart: Disable FS tests and certain test cases for list-incomplete-uploads. 2016-05-28 15:15:53 -07:00
Karthic Rao
1f51af6f37 Listmultipart tests. 2016-05-28 15:15:53 -07:00
Krishna Srinivas
b1e2b7dea2 Fix list-incomplete uploads for XL. 2016-05-28 15:15:53 -07:00
Harshavardhana
34e9ad24aa XL: Introduce new API StorageInfo. (#1770)
This is necessary for calculating the total storage
capacity from object layer. This value is also needed for
browser UI.

Buckets used to carry this information, this patch
deprecates this feature.
2016-05-28 15:15:53 -07:00
Harshavardhana
b2293c2bf4 XL: Rename, cleanup and add more comments. (#1769)
- xl-v1-bucket.go - removes a whole bunch of code.
- {xl-v1,fs-v1}-metadata.go - add a lot of comments and rename functions
   appropriately.
2016-05-28 15:15:53 -07:00
Harshavardhana
553fdb9211 XL: Bring in support for object versions written during writeQuorum. (#1762)
Erasure is initialized as needed depending on the quorum and onlineDisks.
This way we can manage the quorum at the object layer.
2016-05-28 15:15:53 -07:00
Harshavardhana
cae4782973 XL: explicit deleteObject is not needed after rename failure. (#1760)
Reason is renameObject() does deleteObject() upon writeQuorum
failure if not keeps the successfully renamed parts if we have
reached readQuorum.
2016-05-28 15:15:53 -07:00
Krishnan Parthasarathi
3550660163 Return error for empty parts in multipartupload complete (#1758) 2016-05-28 15:15:53 -07:00
Harshavardhana
a4771265cf XL: Abortmultipart should update uploads.json properly. (#1757) 2016-05-28 15:15:53 -07:00
Harshavardhana
a9e778f460 XL/fs: initObjectLayer should cleanup tmpMetaPrefix in parallel. (#1752)
Fixes #1747
2016-05-28 15:15:53 -07:00
Harshavardhana
ee6645f421 XL: Add additional PartNumber variable as part of xl.json (#1750)
This is needed for verification of incoming parts and to
support variadic part uploads. Which should be sorted
properly.

Fixes #1740
2016-05-28 15:15:53 -07:00
Harshavardhana
a97230dd56 XL/erasure: Reset dataBlocks to reduce the memory usage. (#1749)
Fixes #1748
2016-05-28 15:15:53 -07:00
Harshavardhana
1e393c6c5b XL: Add new metadata for checksum. (#1743) 2016-05-28 15:15:53 -07:00
Krishna Srinivas
b38b9fea79 XL/erasure: fix for skipping 0 padding. (#1737)
Fixes #1736
2016-05-28 15:15:53 -07:00
Krishna Srinivas
6d84e84b3c XL/mutltipart: fix partnumber to partname association. (#1739)
Fixes #1738
2016-05-28 15:15:53 -07:00
Harshavardhana
a00a5c6e7e XL: Multipart update uploads.json properly. (#1741) 2016-05-28 15:15:53 -07:00
Harshavardhana
ed43d5e02b No need to delete file inside erasure code (#1732) 2016-05-28 15:15:53 -07:00
Harshavardhana
293d246f95 XL/FS: Rewrite in new format. 2016-05-28 15:15:53 -07:00
Anand Babu (AB) Periasamy
63c65b4635 filter GOPATH from stack trace (#1755) 2016-05-25 02:32:35 -07:00
Harshavardhana
64b0976e1b Remove probe and tasker. (#1733)
Fixes #1717
2016-05-24 18:43:33 -07:00
Aakash Muttineni
b48b2e7f7c Part ID check (#1730)
* Added check in PutObjectPartHandler to make sure part ID does not exceed 10000. ErrInvalidMaxParts written to response if part ID exceeds the maximum value.
2016-05-24 01:52:47 -07:00
Krishnan Parthasarathi
584813e214 Used MINIO_PROFILE_DIR for saving profile information of a minio server (#1722)
To specify the directory where profiling information should be saved
  ```
    export MINIO_PROFILE_DIR=/path/to/profile/dir
  ```
By default, profiling information would be saved in a directory created
using ioutil.TempDir, which would be displayed in stdout on starting the
minio server.
2016-05-22 22:11:39 -07:00
Krishnan Parthasarathi
3a980eac1a Fix shadowing of err variable (#1718) 2016-05-21 00:43:47 -07:00
Harshavardhana
5a4b074ca0 XL: PutObject incorrectly returned after deleting multipart object. (#1715)
Fixes #1714
2016-05-20 17:27:48 -07:00
Harshavardhana
f76d975304 xl: StatVol and ListVols should handle cases when disks are missing. (#1703)
Remove unnecessary code, handle cases of missing disk.
2016-05-20 16:45:53 -07:00
Harshavardhana
6015a7a3cd Add mention-bot config 2016-05-20 13:53:15 -07:00
Krishna Srinivas
b83b87a7f6 XL/Incompleteuploads: list should save the tree-walk go routine to the map if eof is not reached. (#1695) 2016-05-20 12:09:21 -07:00
Krishna Srinivas
5b95f097d4 multipart: listing does not skip uploadIDmarker. (#1708)
Fixes #1706
2016-05-20 11:48:28 -07:00
Harshavardhana
e4240aa58f XL/objects: Initialize format.json outside of erasure. (#1640)
Fixes #1636

New format now generates a UUID and includes it along with
the order of disks. So that UUID is the real order of disks
and on command line user is able to specify disks in any order.

This pre-dominantly solves our dilemma.
```
{
   "format" : "xl",
   "xl" : {
      "version" : "1",
      "disk": "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
      "jbod" : [
         "00e4cf06-5bf5-4bb5-b885-4b2fff4a7959",
         ....
         "c47d2608-5067-4ed7-b1e4-fb81bdbb549f",
         "a543293e-99f1-4310-b540-1e450878e844",
         "18f97cbe-529a-456a-b6d4-0feacf64534d"
      ]
   },
   "version" : "1"
}
```
2016-05-20 02:22:22 -07:00
Andreas Linz
f5dfa895a5 Exit with error code if minio server fails to start (#1704)
This commit replaces the call to `errorIf` with `fatalIf`, so that the
minio server exits with an non-zero exit status if something fails, e.g.
the port was already openend by another process.
2016-05-20 02:05:52 -07:00
Harshavardhana
50c328ff19 XL: RenameFile should rename and cleanup on writeQuorum. (#1702)
Fixes #1683
2016-05-20 01:56:46 -07:00
Anand Babu (AB) Periasamy
b8405ca172 simplify profiler cleanup 2016-05-19 19:19:32 -07:00
Harshavardhana
f6d9e73548 posix: Do not lowercase names, return as is. Object layer will filter them out. (#1699) 2016-05-19 18:52:55 -07:00
Harshavardhana
7ae5470395 XL: simplify isMultipartObject not need to handle unknown errors. (#1686)
Unknown errors are just logged with errorIf.
2016-05-19 17:10:33 -07:00
Harshavardhana
9fdb69563d handler: CopyObject should save metadata. (#1698)
- Content-Type
- Content-Encoding
- ETag

Fixes #1682
2016-05-19 17:10:08 -07:00
Harshavardhana
00d0558131 XL: Enable tests for content-type for Head and Get
Fixes #1674
2016-05-19 15:34:18 -07:00
Bala.FA
13e4618309 XL/fs: Return saved content-type during GetObject
Fixes #1674
2016-05-19 15:34:18 -07:00
Krishnan Parthasarathi
2f05aacbf2 Stop profiling on exit of main goroutine (#1670)
* Stop profiling on exit of main goroutine

Previously, profiling was stopped since Stop() method was called on exit of cli.BeforeFunc.
This lead to profiling to be stopped prematurely.

* Moved profiling switch statement to a separate func
2016-05-19 14:50:54 -07:00
Krishna Srinivas
dc36594ef4 XL/heal: Should skip healing if CreateFile() failed on the part which needed healing. (#1693)
Fixes #1684
2016-05-19 11:32:47 -07:00
Krishna Srinivas
537568f9f9 XL/ListVols: Fix panic. Skip if slice is nil. (#1694)
Fixes #1692
2016-05-19 11:32:19 -07:00
Harshavardhana
62b4fd6964 XL: Close the reader properly. 2016-05-18 20:16:19 -07:00
Harshavardhana
7d6ed50fc2 objects: Save all the incoming metadata properly. (#1688)
For both multipart and single put operation
2016-05-18 19:54:25 -07:00
Frank
af85acf388 Added ETag as an exposed header (required for multi part uploads) (#1681)
* Added ETag as an exposed header (required for multi part uploads)

* Fix formatting on adding ETag for exposed headers
2016-05-18 19:17:32 -07:00
Frank
a4fef436c8 Fix formatting for adding ETag for exposed headers (#1687) 2016-05-18 19:17:17 -07:00
Harshavardhana
404364ba73 XL/fs: ListMultipartUploads should list only requested entries. (#1668)
Fixes #1665
2016-05-18 15:06:29 -07:00
Krishna Srinivas
3c1ef3fee2 Locking: move locking code from xl-erasure to xl-objects. (#1673)
Fixes #1639 #1649 #1650 #1624
2016-05-18 15:05:23 -07:00
Bala FA
a0865122a7 XL/objectLayer: Save additional meta data during PutObject (#1672)
Fixes  #1602
2016-05-18 13:56:11 -07:00
Krishna Srinivas
824c8a39f1 XL/Multipart: If the part is already uploaded ignore the newly uploaded part. (#1677)
Fixes #1644
2016-05-18 13:37:28 -07:00
Krishna Srinivas
71b9341fc7 XL/Multipart: Cleanup uploads.json after abort-multipart-upload. (#1678)
Fixes #1663
2016-05-18 13:30:58 -07:00
Harshavardhana
b69a97aed4 server: Set rLimit properly to the max. (#1676)
4000 is too small to handle 500 go-routines.

Fixes #1666
2016-05-18 11:34:24 -07:00
karthic rao
2da34e4668 vendor changes to pkg/profile (#1671) 2016-05-18 09:22:06 -07:00
Harshavardhana
4bc923e63b XL/fs: Optimize calling isBucketExist() (#1656)
* posix: Avoid using getAllVolumeInfo() in getVolumeDir()

This is necessary compromise to avoid significant slowness this
causes under load. The compromise is also substantial in a way
so that to avoid penalizing common cases v/s special cases.

For buckets with Caps on Unixes, we filter buckets based on the
latest anyways, so this is completely acceptable.

* XL/fs: Change the usage of verification of existance of buckets.

Optimize calling isBucketExists, it is not needed for all call
paths. isBucketExist should be called only for calls which use
temporary volume location for operations, for the rest rely on
the errors returned on their original call path.

Remove usage of filtering as well across all volume names.
2016-05-17 21:22:27 -07:00
Harshavardhana
4214da65af XL/fs: MakeVol replies should be consistent. (#1667)
Fixes #1658
Fixes #1633
2016-05-17 18:41:17 -07:00
Krishnan Parthasarathi
596fe65e84 Write pprof output files under config dir supplied (#1660)
Since config dir, supplied as command line argument, is parsed after pprof
output directory is determined, pprof output files are  written in ~/.minio/profile
directory instead of <configDir>/profile/. This change fixes this behaviour.
2016-05-17 11:44:40 -07:00
Krishna Srinivas
39865c0d2e XL/Multipart: Fix list multipart output. (#1661)
Fixes #1541
2016-05-17 11:44:32 -07:00
Harshavardhana
1760687c83 XL: Make sure to create proper temporary files for renames to succeed. (#1654)
Renames work in a special manner, temporary location files should
be created properly.

Fixes #1653
Fixes #1651
2016-05-16 15:40:57 -07:00
Harshavardhana
9472299308 logging: Log only for unhandled errors, remove all the debug logging. (#1652)
This patch brings in the removal of debug logging altogether, instead
we bring in the functionality of being able to trace the errors properly
pointing back to the origination of the problem.

To enable tracing you need to enable "MINIO_TRACE" set to "1" or "true"
environment variable which would print back traces whenever there is an
error which is unhandled or at the handler layer.

By default this tracing is turned off and only user level logging is
provided.
2016-05-16 14:31:28 -07:00
Harshavardhana
8828fd1e5c vendor: Remove unused packages. 2016-05-15 00:04:56 -07:00
Harshavardhana
7de206cb85 XL: ListVols should provide consistent view. (#1648)
Additionally get list of all volumes in parallel for aggregation
and quorum verification.

Fixes #1647
2016-05-14 23:57:57 -07:00
Harshavardhana
498ce1e9bb handler: Add a waitgroup to avoid expect100Continue crash. (#1623)
This waitgroup allows for safe blocking operation where we can cleanly
control the flow of the writes and the underlying pipe altogether.

Fixes #1553
2016-05-14 17:18:00 -07:00
Harshavardhana
5b29cefd40 api: DeleteObject should always return 204. (#1645)
Fixes #1643
2016-05-14 15:47:19 -07:00
Hori Ryota
e03ebfd13b Add default cmd (#1625) 2016-05-14 02:58:50 -07:00
Harshavardhana
74c23a3544 docs: Move developer docs from top-level to its own directory. (#1642) 2016-05-14 02:47:16 -07:00
Harshavardhana
025054fb36 XL: CreateFile/ReadFile should write and read from all disks in parallel. (#1612)
* XL: CreateFile should write to all disks in parallel.

* XL: ReadFile should read from all disks in parallel.
2016-05-14 01:57:04 -07:00
Bala FA
7264cd2ab3 Fix error message when wrong set of disks are passed (#1634)
Previously when wrong set of disks are given with last minio server
run, it throws unclear error message.  This is fixed by returning
appropriate errors.

Fixes #1591
2016-05-13 23:04:10 -07:00
Harshavardhana
0e4e9c4bc1 XL: ListDir should return each List from a random disk in the set. (#1613)
Fixes #1609
2016-05-13 18:12:26 -07:00
Krishna Srinivas
8099396ff0 xl/putObject: Should take care of the situation if an object already exists at the location. (#1606)
Fixes  #1598 #1594 #1595
2016-05-13 11:52:36 -07:00
Krishna Srinivas
d267696110 Validation: Reject object names with trailing "/". (#1619)
Fixes #1616
2016-05-13 11:43:06 -07:00
Bala FA
43539a0c86 posix: parseDirents() should follow symlink and get values. (#1631)
Previously parseDirents() ignores symbolic links.  This patch fixes
the issue by following the symlink using os.Stat().

Fixes #1545
2016-05-13 11:39:48 -07:00
Krishnan Parthasarathi
9e45d138cc Closed readCloser for each multipart in xl.GetObject (#1629)
* Closed readCloser for each multipart in xl.GetObject
2016-05-13 04:50:13 -07:00
karthic rao
ee8605e333 Make bucket failure fix with high concurrent load (#1630) 2016-05-13 04:03:38 -07:00
karthic rao
e4958f9757 Removing regexp check and adding string based check, regexp check was unnecessary here (#1627) 2016-05-13 03:33:53 -07:00
Harshavardhana
b62774d32f storage/xl: Return errVolumeAccessDenied if disks cannot be accessed. (#1621)
Fixes #1614
2016-05-12 21:01:11 -07:00
koolhead17
d6e0f3ab33 added awscli commands & minor typo fix. (#1578) 2016-05-12 16:35:11 -07:00
Bala FA
3ff0a56e62 XL: Ignore errDiskNotFound in certain situations (#1610)
When a disk is removed while an operation is going on
(eg. single/multipart put object, list/multipart list objects etc),
its required to ignore errDiskNotFound error and continue the
operation.

Fixes #1552
2016-05-11 23:42:14 -07:00
Harshavardhana
50431e91a6 erasure: Handle failed disks so that we initialize properly if they are missing. (#1607)
Fixes #1592
Fixes #1579
2016-05-11 18:58:32 -07:00
Harshavardhana
d4745c7d6a object: PutObjectHandler should set the md5Sum properly. (#1604)
Additionally add a test case as well for validating for us
to reply BadDigest properly.

Fixes #1603
2016-05-11 16:13:37 -07:00
Bala FA
adbcafefad xl/CreateFile: handle errFileNameTooLong error properly (#1523)
When errFileNameTooLong error is returned from posix, xl.CreateFile()
treats the error specially by returning the same error immediately.

Fixes #1501
2016-05-11 12:55:02 -07:00
Harshavardhana
86e5d71519 erasure: MakeVol, DeleteVol and StatVol should hold locks. (#1597)
Since there is a good amount of overlap, each code has to lock
properly for the operation they are going to perform.

- MakeVol create vols in a routine on all disks, hold locks.
- DeleteVol delete vols in a routine on all disks, hold locks.
- StatVol stat vols in a routine on all disks, hold locks.

Fixes #1588
2016-05-11 12:54:21 -07:00
Harshavardhana
72748d2073 erasure: healVolume err should be different from shadowed version. (#1590)
Multiple go-routines updating the same shadowed variable can
cause a data race, avoid it by using its own err variable.

Fixes #1589
2016-05-11 01:36:09 -07:00
Harshavardhana
49141eb3e4 http: Remove minhttp package and use standard Golang http. (#1587)
The functionality provided by minhttp will be implemented
cleanly through our own APIs. Since we are not going to
send SIGUSR2 and manage configuration in that manner, it
doesn't make sense to use minhttp.

Fixes #1586
2016-05-10 18:03:00 -07:00
Harshavardhana
d1fa1d9352 Remove binary files from previous commit. 2016-05-10 15:49:17 -07:00
karthic rao
26e2c4bf4d Replacing fastsha256 with crypto/sha256 package from golang standard package (#1584) 2016-05-10 14:20:11 -07:00
Krishna Srinivas
b044336329 XL/GetObject: If the offset does not fall in the first "dataBlock" it gives incorrect data. (#1583)
Fixes #1582
2016-05-10 11:38:49 -07:00
Krishna Srinivas
e99cb05516 XL/GetObject: offset should be reset to 0 after reading first part. (#1580) (#1581) 2016-05-10 10:38:12 -07:00
Krishna Srinivas
409e09c1e5 XL/Selfheal: skip reading from disk if ReadFile had returned error. (#1575) 2016-05-10 01:24:58 -07:00
Krishna Srinivas
c314a98c1a XL/list: fix panic on list when a disk is down. (#1562) 2016-05-10 00:35:29 -07:00
Harshavardhana
5f0ca64346 erasure: listOnlineDisks should return errFileNotFound for errReadQuorum. (#1573)
Fixes #1571
2016-05-10 00:10:34 -07:00
Harshavardhana
0db3218d5d xl: getPartsMetadata fetch parts and decode in go-routine. (#1569)
Ref #1516
2016-05-09 23:51:05 -07:00
Harshavardhana
eec41c369c posix: Return diskNotFound error rather than errVolumeNotFound (#1568)
Fixes #1559
2016-05-09 18:57:39 -07:00
Harshavardhana
b66c3bf35e server: Enable server profiling as needed. (#1565) 2016-05-09 16:18:56 -07:00
Harshavardhana
f733120d3d xl: CompleteMultipartUpload make sure to delete uploads.json (#1539)
Fixes #1537

Ref #1540 - for missing functionality in this patch.
2016-05-09 12:09:48 -07:00
Krishna Srinivas
6627388dc3 posix: remove dead code related to posix reserved suffixes. (#1555) 2016-05-09 11:40:30 -07:00
Harshavardhana
9d41414fb5 posix: reserved files should be filtered out at posix not object layer. (#1554) 2016-05-09 02:53:08 -07:00
Harshavardhana
722abe2d0f xl/fs: pathJoin now takes variadic inputs. (#1550)
Retains slash for the last element.

Fixes #1546
2016-05-09 00:46:54 -07:00
Krishna Srinivas
04a5b25929 Multipart: Minimum part size limit does not apply to the last part during CompleteMultipartUpload. (#1518) (#1538) 2016-05-08 23:49:49 -07:00
Harshavardhana
90ea494338 erasure: waitCloser should implement CloseWithError. (#1543)
This is needed so that the other end of the pipe receives
and error, cleanups temporary files.
2016-05-08 16:26:10 -07:00
Harshavardhana
a8fdd04e62 erasure: ReadFile should honor proper offsets. (#1542)
Fixes #1535
2016-05-08 15:39:24 -07:00
Harshavardhana
76c511c9fe api: Extend S3 errors with Minio errors. (#1533)
Fixes #1530
2016-05-08 12:36:16 -07:00
Krishna Srinivas
75320f70d0 multipart: reject part upload if size is less than 5MB. (#1518) 2016-05-08 12:06:05 -07:00
Krishna Srinivas
88e1c04259 XL/ListDir: break out of loop if list on one disk is a success. (#1534) 2016-05-08 12:05:19 -07:00
Krishna Srinivas
a205aca6d2 init: Cleanup .minio/tmp directories recursively. Also takes care of cleaning up of parts directory during abortMultipartUpload. (#1532) 2016-05-08 10:15:34 -07:00
Harshavardhana
3f51dd4fd4 xl: CompleteMultipartUpload should rename files in a routine. (#1527)
This solves the client timeout while renaming 9000+ parts.

Fixes #1526
2016-05-08 02:38:35 -07:00
Harshavardhana
56b7df90e1 xl/fs: ListObjectParts should set nextPartNumberMarker properly. (#1528)
For list requests on parts more than 1000, would lead to an infinite
loop.

Fixes #1522
2016-05-08 02:21:12 -07:00
Harshavardhana
a56d5ef415 xl/fs: isFunctions should only return boolean. (#1525)
log the unrecognize errors.
2016-05-08 01:58:05 -07:00
Harshavardhana
937d68202d server: Deadcode removal. (#1517) 2016-05-07 21:47:33 -07:00
Harshavardhana
bf563afb80 xl: DeleteObject regression over errChannel. (#1521)
DeleteObject would hang indefinitely - fix it.
2016-05-07 12:48:12 -07:00
Harshavardhana
091c1e8456 copyObject: No need to verify md5sum. (#1520)
Multipart objects are kept in non hex md5sum format.
This format doesn't comply with hex, so decoding
would fail invariably.

This is not necessary to validate and its not expected
error during a CopyObject operation.

Fixes #1519
2016-05-07 03:43:08 -07:00
Harshavardhana
751fa972f5 xl/fs: Multipart re-org introduce "uploads.json" (#1505)
Fixes #1457
2016-05-07 02:08:03 -07:00
Harshavardhana
434423de89 xl: Move format detection inside xl objects. (#1515)
Fixes #1449
2016-05-07 00:59:43 -07:00
Harshavardhana
a20ccb1e83 server: Print proper endpoint, along with https if configured. (#1514)
Fixes #1492
2016-05-06 21:18:29 -07:00
Harshavardhana
0625c050e6 xl/tests: Enable server handler tests over XL. (#1512)
Fixes #1513
2016-05-06 16:47:23 -07:00
Harshavardhana
0b74f5624e xl: Fix how we deal with read offsets at erasure layer. (#1511)
Requires skipping necessary parts of dataBlocks during
decoding phase and requires us to properly skip the
entries as needed.

Thanks to Karthic for reproducing this important issue.

Fixes #1503
2016-05-06 16:25:08 -07:00
Krishna Srinivas
c06b9abc15 bucket-handlers: do not unescape marker as gorilla layer would have already done it. (#1495) (#1510) 2016-05-06 16:04:46 -07:00
Krishna Srinivas
a5d31d4254 XL/ListObjects: use string.TrimSuffix instead of Trim. (#1498) (#1509) 2016-05-06 13:28:55 -07:00
karthic rao
20ca65c793 Cleanup: mispell fixes 2016-05-06 12:32:44 -07:00
karthic rao
0b4bbe6d9e Adding XL Object layer validation for existing unit tests of single node (#1499)
object layer.

Adding isBucketExist check for GetObjectInfo in the XL layer.
2016-05-06 11:57:04 -07:00
Krishna Srinivas
48d3be36da XL/ListObjects: Fix ordering issue during listing if the files were uploaded as multipart uploads. (#1498) (#1506)
i.e if two files "tmp" and "tmp.1" are uploaded as multipart we would list ""tmp.1"" before ""tmp"" as "tmp.1/" < "tmp/"
2016-05-06 10:19:09 -07:00
Harshavardhana
5133ea50bd xl/fs: Make i/o operations atomic. (#1496) 2016-05-05 20:28:22 -07:00
Harshavardhana
17868ccd7f handlers: overhaul entire writErrorResponse, simplify. (#1472) 2016-05-05 20:24:29 -07:00
Harshavardhana
ba5805e60a bucketPolicy: Do not use regexes, just do prefix matches. (#1497)
AWS arn supports wildcards and this is flat namespace, simple
prefix matching is fine.

Fixes #1481
Fixes #1482
2016-05-05 19:58:48 -07:00
Harshavardhana
ca097de96c xl/fs: Add initObjectLayer function. (#1494)
Fixes #1493
2016-05-05 15:00:03 -07:00
Bala FA
658a595d7a xl-erasure: RenameFile should support quorum. (#1487)
Fixes #1463
2016-05-05 13:03:53 -07:00
Krishna Srinivas
247e835d7b object: move go-routine listing from posix to objectLayer. (#1491) 2016-05-05 12:51:56 -07:00
Harshavardhana
46680788f9 xl/fs: cleanup '/.minio/tmp' directory on each initialization. (#1490) 2016-05-05 01:54:43 -07:00
Harshavardhana
ad40036cba posix: filepath shouldn't be used anymore use path.Join (#1486) 2016-05-05 01:39:26 -07:00
Harshavardhana
82fbe908a3 object: DeleteBucket should return proper error for BucketNotEmpty. (#1489)
Fixes #1488
2016-05-05 01:35:21 -07:00
Harshavardhana
f145e1042f quick/config: No need to use Data() with type assertion. (#1480)
Since input to quick.New() is a pointer the unmarshalled value
internally already has the value, subsequent type assertions
are not needed.

Thanks to Bala for finding this behavior.

Fixes #1475
2016-05-04 20:22:15 -07:00
Rajiv Makhijani
9dccbd6478 New Dockerfile for building & running minio inside Docker (inc. autobuild support) (#1473) (#1485) 2016-05-04 17:07:19 -07:00
karthic rao
82113b747c Resource matching fix to overcome issues with regular expression based match (#1476) 2016-05-04 16:56:57 -07:00
Rajiv Makhijani
a5959789d5 Make minimum file space percent a constant (#1484) 2016-05-04 15:39:06 -07:00
Harshavardhana
6988ed9257 xl/getObjectInfo: Returns back proper size, modTime and md5Sum. (#1479)
Fixes #1469
2016-05-04 15:28:58 -07:00
Rajiv Makhijani
321aefa026 Add Response for PostPolicyBucketHandler (#1477) (#1483) 2016-05-04 15:24:10 -07:00
Harshavardhana
dd417e5476 fs: Handle cases of PutObject for an existing prefix. (#1478) 2016-05-04 12:18:40 -07:00
Bala FA
da3a53376c server: save and compare multiple disks are used (#1474)
When server is run with multiple disks which uses xl interface where
order and count of disks are important, this patch saves such disks
configuration and compares in next run if there is a mismatch.

Fixes #1458
2016-05-04 12:18:20 -07:00
Harshavardhana
e4d89d8156 xl/deleteObject: Support deleting special multipart object. (#1470)
Fixes #1452
2016-05-03 17:52:54 -07:00
Harshavardhana
d0e854afb7 xl/fs: Bring in ".minio/tmp" directory support. (#1464)
All transactions happen through this directory inside ".minio/temp".
Adding this allows us to remove any temporary files which were not
committed before.

Fixes #1462
Fixes #1444
2016-05-03 16:10:24 -07:00
Harshavardhana
6f1811ee4d config: Migration should save region properly. (#1468)
Fixes #1466
2016-05-03 15:17:58 -07:00
Yurii
bba5468368 minio: Replace 'bucket already exists' error by 'bucket already owned by you'. (#1465)
S3 API returns BucketAlreadyExists error when some another user has such bucket.
If user that creates the bucket already has it, s3 returns BucketAlreadyOwnedByYou.
As minio has only one user, it should behave accordingly.
Otherwise it causes failures in the applications that ignore creation of already existing bucket in the account, but fail when bucket name is used by someone else.
2016-05-03 03:19:04 -07:00
Harshavardhana
7ae40eb1bb minhttp: Remove probe usage, move to golang error. (#1459)
Fixes #1454
2016-05-03 01:07:34 -07:00
Harshavardhana
ad8e27a963 xl: Rename 'xl.json' to 'file.json' (#1461)
Fixes #1460
2016-05-02 17:42:01 -07:00
Harshavardhana
ac7a7cec20 bucket-policy: Delete policy should remove policy properly. (#1456) 2016-05-02 16:58:10 -07:00
Harshavardhana
afd59c45dc xl/fs: Move few functions into common code. (#1453)
- PutObject()
- PutObjectPart()
- NewMultipartUpload()
- AbortMultipartUpload()

Implementations across both FS and XL object layer
share common implementation.
2016-05-02 16:57:31 -07:00
Harshavardhana
3bf3d18f1f rpc/client: Implement RenameFile properly. (#1443) 2016-05-02 03:12:18 -07:00
Harshavardhana
8102a4712a xl/metadata: Keep the json erasure tag consistent. (#1447)
Currently the on-disk json has "Erasure" we should
keep it consistent name and move to lower case instead.
2016-05-02 01:13:07 -07:00
karthic rao
2393a3a0be XL non-recursive fix (#1450) 2016-05-01 23:16:44 -07:00
Harshavardhana
d006129fb5 xl/vol: Add healing and quorum support for StatVol, MakeVol.
Fixes #1437
2016-05-01 18:42:00 -07:00
Harshavardhana
7caa82f32f xl/fs: Rename minioMetaVolume to minioMetaBucket. (#1442) 2016-05-01 18:13:10 -07:00
Krishna Srinivas
286de4de2c XL - fixes mostly related to multipart listing. (#1441)
* XL/Multipart: Use json.NewDecoder to decode read stream.

* XL/Multipart: fix recursive and non-recursive listing.

* XL/Multipart: Create object part with md5sum later using RenameFile.

* XL/Multipart: ListObjectParts should list parts in order.

previously: uploadID.10.md5sum < uploadID.2.md5sum
fix       : uploadID.00010.md5sum > uploadID.00002.md5sum

* XL/Multipart: Keep the size of each part in the multipart metadata file to avoid stats on the parts.

* XL/Multipart: fix listing bug which was showing size of the multipart uploaded objects as 0.
2016-05-01 17:52:16 -07:00
Harshavardhana
ba7a55c321 xl: ReedSolomon code fix small file erasure bug. (#1431)
For files less than 'dataBlocks', erasure encoding would fail
with short data due to a bug in the implementation itself.

Relax the error return, even a single byte can be properly
erasure coded without issues.

Fixes #1413
2016-05-01 15:30:40 -07:00
Harshavardhana
e05aa762a9 fs: Create object part with md5sum later using RenameFile. (#1440)
Fixes #1340.
2016-05-01 14:50:30 -07:00
Krishna Srinivas
0c27d8e5b1 XL/Multipart: maintain the parts info in multipart.json after complete-multipart-upload. (#1436) 2016-05-01 01:25:48 -07:00
Harshavardhana
443ec37765 xl: Add disk usages properly for ListVols() and StatVol(). (#1435) 2016-04-30 23:34:43 -07:00
Bala FA
d5df8b8b8d xl: remove unused err return in listFileVersions() (#1434) 2016-04-30 03:21:54 -07:00
Harshavardhana
ac2933c799 windows: Enable erasure test for windows. (#1432)
Fixes #1363
2016-04-30 02:52:23 -07:00
Bala FA
84afec9ae0 xl: fix DeleteFile() removing meta data files without updating it (#1433)
Fixes #1428 #1427
2016-04-30 02:52:15 -07:00
Harshavardhana
27c50a70cc obj: support object names with curly braces. (#1429)
Example files like

```
/usr/share/mozilla/extensions/{ec8030f7-c20a-464f-9b0e-13a3a9e97384}/ubufox@ubuntu.com.xpi
Song's Son.ogg
```

Should be supported.
2016-04-29 20:19:08 -07:00
Bala FA
a978975eea xl: add quorum support for DeleteFile() (#1426)
Fixes #1396
2016-04-29 18:43:18 -07:00
Harshavardhana
dc45ea3946 log: Fix file logging, enable it properly. (#1424)
Fixes #1419
2016-04-29 17:54:02 -07:00
Harshavardhana
9eb56f0676 xl/healFile: Handle errors and continue (#1425)
Fixes #1354
2016-04-29 17:52:49 -07:00
Harshavardhana
10a010c1ad xl/fs: Object layer - keep common functions into single place. (#1423) 2016-04-29 17:52:17 -07:00
Harshavardhana
a9935f886c vendor: update reedsolomon package with new perm improvements. (#1422) 2016-04-29 14:59:03 -07:00
Harshavardhana
4e34e03dd4 xl/fs: Split object layer into interface. (#1415) 2016-04-29 14:24:10 -07:00
nomadlogic
4d1b3d5e9a docs: FreeBSD minio source intructions (#1421)
* Modifications of documentation for using and building minio server on FreeBSD.

- update example of enabling compression to use lz4 vs gzip and provide
  explanation of benefits of lz4

- provide walkthrough of building minio server on FreeBSD with binary
  golang and gmake

* Fixing markdown syntax for code blocks so we render correctly.

* typo fix

* reword compression enablement docs for easier reading
2016-04-29 13:35:20 -07:00
Krishna Srinivas
7066ce5160 XL/Multipart: rename the parts instead of concatenating. (#1416) 2016-04-29 12:17:48 -07:00
Krishna Srinivas
39df425b2a lock: bug fixes. (#1420)
* release Lock on map before trying to Lock NS
* delete NS lock from map if no more refs.
* refactor to avoid repetition of code.
2016-04-29 11:39:20 -07:00
Harshavardhana
984903cce1 server: Add global namespace lock. (#1398)
Fixes #1393
2016-04-29 01:29:09 -07:00
karthic rao
8deddb82f4 Cleanup: Moving IsValidLocationContraint to handler utils 2016-04-28 20:01:11 -07:00
Harshavardhana
a1a667ae5d xl: Change fileMetadata to xlMetadata. (#1404)
Finalized backend format

```
{
    "version": "1.0.0",
    "stat": {
        "size": 24256,
        "modTime": "2016-04-28T00:11:37.843Z"
    },
    "erasure": {
        "data": 5,
        "parity": 5,
        "blockSize": 4194304
    ],
    "minio": {
        "release": "RELEASE.2016-04-28T00-09-47Z"
    }
}
```
2016-04-28 19:27:02 -07:00
Harshavardhana
41b35cff7b xl: Fixes a bug in read quorum ListFiles() (#1412)
Fixes a bug in #1406
2016-04-28 17:32:46 -07:00
Harshavardhana
eed756777b object: Allow '[' and ']' as part of object names. 2016-04-28 13:29:32 -07:00
Harshavardhana
2ac10209cc xl: ListFiles - return sorted files. (#1408)
Fixes #1407
2016-04-28 01:48:57 -07:00
Bala FA
5bd6b0b510 xl: check read quorum for ListFiles() (#1406)
Fixes #1364
2016-04-27 21:09:26 -07:00
karthic rao
1813e9c070 Cleanup - Comments and readability fixes (#1386) 2016-04-27 19:28:13 -07:00
Scott McClellan
f87a19a15c Minor changes to CONTIRBUTING.md instructions (#1403) 2016-04-27 16:57:16 -07:00
Harshavardhana
5fffd558d0 xl/heal: Make healFile non-blocking for StatFile and ReadFile. (#1399)
Fixes #1355
2016-04-27 15:10:19 -07:00
Harshavardhana
b51bef85e6 objectapi: ListMultipart now lists more than 1000 entries in non-recursive. (#1397)
Having keyMarker with "/" is a valid marker.

Fixes #1394
2016-04-27 13:19:48 -07:00
Krishna Srinivas
d0e5470050 ListMultipart fixes (#1392)
* ListMultipart: listLeafEntries() - return earlier if a directory is found.
* ListMultipart: do listLeafEntries() only for directories.
2016-04-27 00:15:40 -07:00
Harshavardhana
90987df9b4 objectapi: Simplify ListMultipart combine recursive and non-recursive. (#1390)
Fixes #1365
2016-04-26 17:57:16 -07:00
Harshavardhana
ad1abc4486 xl-v1/Cleanup: use listOnlineDisks instead of getReadableDisks. (#1389)
Remove usage of getFileVersionQuorumMap, instead use listFileVersions
to get the version list and extract higherVersion.

Fixes #1379
Fixes #1378
Fixes #1377
2016-04-26 13:03:37 -07:00
Krishna Srinivas
4333e529e6 xl/ListFiles: return as many objects as requested. (#1383)
* xl/ListFiles: return as many objects as requested and take care of eof (#1361)

* xl/ListFiles: fix review comments.

* xl/ListFiles: Add windows filepath translation.

* xl/ListFiles: Use slashSeparator instead of "/". Remove filepath.FromSlash() as golang-windows takes care of it automatically.
2016-04-26 10:35:39 -07:00
koolhead17
9685f88b84 Added FreeBSD installation steps with ZFS. (#1388) 2016-04-26 10:17:02 -07:00
Harshavardhana
5f80edf232 routers: Fix a crash while initializing network fs. (#1382)
Crash happens when 'minio server filename' a file name is
provided instead of a directory on command line argument.

```
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
panic(0x5eb460, 0xc82000e0b0)
	/usr/local/opt/go/libexec/src/runtime/panic.go:464 +0x3e6
main.splitNetPath(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:49 +0xb7
main.newNetworkFS(0x7fff5fbff9bd, 0x7, 0x0, 0x0, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/network-fs.go:90 +0x20a
main.configureServerHandler(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x0, 0x0)
	/Users/harsha/mygo/src/github.com/minio/minio/routers.go:43 +0x6ce
main.configureServer(0xc82024e1c8, 0x5, 0xc8200640e0, 0x1, 0x1, 0x5)
	/Users/harsha/mygo/src/github.com/minio/minio/server-main.go:86 +0x67
```
2016-04-25 18:10:40 -07:00
Harshavardhana
42254b5c4d xl: Rename blockingWriteCloser to waitCloser. (#1376) 2016-04-25 16:00:58 -07:00
Harshavardhana
00c697393a Merge pull request #1381 from minio/xl-layer
Implement XL layer
2016-04-25 15:59:30 -07:00
Harshavardhana
55032ffdf9 xl: Simplify blockingWriter and its usage. (#1373)
This removes odd races since we don't need to
track errors and avoids locking. All we need
is a Wait() and Done() waitgroup.
2016-04-25 12:47:31 -07:00
Harshavardhana
8bce699dae xl: Add logging. (#1372) 2016-04-25 12:47:31 -07:00
Harshavardhana
57f35c2bcc xl: Introduce new blocking writer to make CreateFile atomic. (#1362)
Creates a new write closer that must be released
by the read consumer. This is necessary so that
while commiting the underlying writers in erasure
coding we need to make sure we reply success only if
we have committed to disk.

This in turn also fixes plethora of bugs related to
subsequent PutObject() races with namespace locking.

This patch also enables most of the tests, other than
ListObjects paging which has some issues still.

Fixes #1358, #1360
2016-04-25 12:47:31 -07:00
Harshavardhana
cab6805f09 xl: Enable a subset of tests for XL branch. (#1359) 2016-04-25 12:47:31 -07:00
Krishna Srinivas
8c85815106 xl: refactor functions to xl-v1-common.go xl-v1-utils.go. (#1357) 2016-04-25 12:47:31 -07:00
Krishna Srinivas
becc814531 Xl layer selfheal quorum2
* xl/selfheal: selfheal based on read quorum on GET

* xl: getReadableDisks() also returns whether self-heal is needed so that this info can be used by ReadFile/SelfHeal/StatFile.

* xl: trigger selfheal from StatFile.
2016-04-25 12:47:31 -07:00
Harshavardhana
9bd9441107 xl: Simplify reading metadata and add a new fileMetadata type. (#1346) 2016-04-25 12:47:31 -07:00
Harshavardhana
f3784d1087 xl: Handle read quorum for StatVol, ListVols 2016-04-25 12:47:31 -07:00
Harshavardhana
91588209fa obj: Object api handle all errors in common location. (#1343) 2016-04-25 12:47:31 -07:00
Krishna Srinivas
5c33b68318 xl: code refactor, cleanup ReadFile and CreateFile. 2016-04-25 12:47:31 -07:00
Bala FA
45b3d3e21f xl: add quorum support for create file 2016-04-25 12:47:31 -07:00
Harshavardhana
141a44bfbf xl: Fix ReadFile to keep the order always for reading the data back. (#1339)
Also fixes a stackoverflow bug in namespace locking.
2016-04-25 12:47:31 -07:00
Harshavardhana
c7bf471c9e list/xl: Fix the way marker is handled in leafDirectory verification. 2016-04-25 12:47:31 -07:00
Krishna Srinivas
c302875774 selfheal: implement self-heal. Heals the missing parts. (#1335) 2016-04-25 12:47:31 -07:00
Harshavardhana
b76f3f1d62 xl: Add more fixes and cleanup.
Simplify cleanup of temporary files during createFile operations.
2016-04-25 12:47:31 -07:00
Bala FA
ada0f82b9a xl: add quorum support for read file and name space locking. (#1333) 2016-04-25 12:47:31 -07:00
Harshavardhana
a98a7fb1ad Implement XL layer - preliminary work. 2016-04-25 12:47:31 -07:00
Harshavardhana
bf8a9702a4 tests: Fix a bug in TestObjectAPIIsUploadIDExists. (#1375)
The following code crashes when upload ID does not
exist, since we are setting err == nil when we find
err == errFileNotFound.

```
if e == nil {
   t.Fatal(e.Error())
```

Fix it.
2016-04-25 12:47:08 -07:00
karthic rao
6e372f83b4 Tests: object api multipart tests and bug fixes. 2016-04-25 10:39:28 -07:00
Harshavardhana
e9fba04b36 logging: Enable logging across storage fs layer. (#1367)
Adds log.Debugf at all the layer - fixes #1074
2016-04-24 00:36:00 -07:00
Harshavardhana
d63d17012d tests: Add API suite tests back for object api. (#1352) 2016-04-21 23:40:01 -07:00
Harshavardhana
444d1f8a65 miniobrowser: Vendorize to new changes in miniobrowser. 2016-04-21 20:35:48 -07:00
karthic rao
560c3bd153 Adding return statement after error response in the lastest commit to verify location constraint (#1348) 2016-04-21 20:08:08 -07:00
Harshavardhana
4cf73caf02 api: Add diskInfo as part of StatVol and ListVols. (#1349)
It is the bucket and volumes which needs to have this
value rather than the DiskInfo API itself. Eventually
this can be extended to show disk usage per
Buckets/Volumes whenever we have that functionality.

For now since buckets/volumes are thinly provisioned
this is the right approach.
2016-04-21 20:07:47 -07:00
Harshavardhana
1284ecc6f2 api: Fix verification of checkLeafDirectory. (#1347)
This fixes a problem where leaf directory has more than 1000
entries, also resulting in listing issues, leading to an infinite
loop.

Fixes #1334
2016-04-21 18:05:26 -07:00
karthic rao
cb1116725b api: verify Location constraint for make bucket. (#1342) 2016-04-20 17:35:38 -07:00
koolhead17
c3d0a3d51e Update README.md
Our community contributor bought this to our attention so we have to add region as well in s3cmd config file.
2016-04-19 10:22:40 -07:00
Harshavardhana
e0f8fed011 object: handle Error responses and handle errDiskFull. (#1331) 2016-04-19 02:42:10 -07:00
Harshavardhana
6bc17a3aea server: Attempt to increase max open files. (#1328) 2016-04-18 22:05:32 -07:00
Dave Henderson
ff9a6b00cc isContainerized should look for /.dockerenv not /.dockerinit
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2016-04-18 19:26:56 -07:00
matt robinson
af907a35a9 add environment var to explicitly indicate containerized and allow running as root (#1327) 2016-04-18 16:30:11 -07:00
Harshavardhana
b47d722d8e fs: Fix filtering out valid paths from previous #1321 fix (#1323)
Fixes #1324
2016-04-17 12:00:23 -07:00
Harshavardhana
33633fd15d fs: Filter out valid paths and volnames. (#1321) 2016-04-16 19:33:29 -07:00
Harshavardhana
b2ec7da9f9 Merge pull request #1314 from minio/split-fs
Split fs code into storage layer and object layer.
2016-04-16 18:09:31 -07:00
Harshavardhana
be002ac01e fs/object: Fix issues from review comments. 2016-04-16 17:57:14 -07:00
Krishna Srinivas
149c6ca094 listMultipart: bugfixes. (#1318) 2016-04-16 16:25:53 -07:00
Harshavardhana
8457af5708 fs: Add proper volume and path validation. 2016-04-16 16:25:53 -07:00
Krishna Srinivas
caa35f68fa listMultipart: implement support for marker. (#1313) 2016-04-16 16:25:53 -07:00
Harshavardhana
30b0b4deba storage/server/client: Enable storage server, enable client storage. 2016-04-16 16:25:53 -07:00
Krishna Srinivas
01a439f95b refactor: add multipart code to the object layer. 2016-04-16 16:25:53 -07:00
Krishna Srinivas
3c48537f20 refactor: refactor code to separate fs into object-layer and fs layer. (#1305) 2016-04-16 16:25:53 -07:00
karthic rao
188bb92d8a bucket-policy parset tests, and bug fixes (#1317) 2016-04-15 18:23:19 -07:00
Harshavardhana
6b3fc03701 docker: Update docker document 2016-04-15 17:40:56 -07:00
Harshavardhana
8112291d43 Add FreeBSD binary link and make a release 2016-04-14 17:22:47 -07:00
Harshavardhana
93666827f4 release: Add freebsd/amd64 build and remove zip, tgz. (#1316) 2016-04-13 23:34:55 -07:00
GarimaKapoor
ac30bef72a Revised Docker.md (#1311) 2016-04-11 23:59:12 -07:00
Bala FA
bea6f33b08 backend/fs: remove timer channel from scanMultipartDir() (#1310)
Previously scanMultipartDir() returns object info channel and timer
channel where timer channel is used to check whether object info
channel is alive or not.  This causes a race condition that timeout
may occur while object info channel in use.

This patch fixes the issue by removing timer channel and uses object
info channel directly where each object info has End bool field
indicates whether received object info is end or not.
2016-04-11 19:00:08 -07:00
Michael Werle
9fb1c79456 Improved Docker examples (#1308)
- Fixed a bug in the persistent docker command ("server" in place of "export")
- Added example of how to set consistent keys with ephemeral data, particularly useful for testing.
2016-04-11 12:33:34 -07:00
Harshavardhana
6b5699b15f config: console logging should be enabled by default. (#1307) 2016-04-09 14:19:59 -07:00
Harshavardhana
33cd910d3a backend/fs: More cleanup and start using checkBuckeArg. (#1306)
backend/fs: More cleanup and start using checkBucketArg.
2016-04-08 17:13:16 -07:00
Bala FA
6af761c86c enhance multipart functions to use fsDirent (#1304)
* backend/fs: scanMulitpartDir returns directories only for recursive listing

* backend/fs: enhance multipart functions to use fsDirent
2016-04-08 11:46:03 -07:00
Matthew Buckett
bedd867c0b The fs command doesn't work any more, using server
Also there's no /export/data folder which stops minio starting so using /export instead.
2016-04-08 08:53:55 -07:00
Harshavardhana
37330bda98 Merge pull request #1302 from harshavardhana/web-api
web: Change /rpc to /webrpc
2016-04-08 02:04:06 -07:00
Harshavardhana
8603185f2f browser: Add new ui-assets.go 2016-04-08 01:47:30 -07:00
Harshavardhana
fbd02d530d web: Change /rpc to /webrpc 2016-04-08 01:46:46 -07:00
Harshavardhana
bad2f2afbb storage: Add storage interface. 2016-04-07 19:33:06 -07:00
Harshavardhana
b182e94acc signature: Handle presigned payload if set.
Validate payload with incoming content.



Fixes #1288
2016-04-07 03:04:18 -07:00
Anand Babu (AB) Periasamy
4e6c4da518 Update README.md 2016-04-06 20:54:06 -07:00
Donald Guy
e8cd1aad8d accessPolicy: prevent backdoor ListBucket via brute-force 404s, per docs + small fixes
* accessPolicy: copy object should require PutObject

* accessPolicy: cite mpu perms doc only for relevant operations

* accessPolicy: prevent backdoor ListBucket via brute-force 404s, per docs
2016-04-06 18:31:40 -07:00
Donald Guy
8b4a5f07b4 accessPolicy: allow anonymous HEAD for Getable objects
* accessPolicy: allow anonymous HEAD for Getable objects

* accessPolicy: allow anonymous HEAD of Listable Buckets
2016-04-06 16:40:54 -07:00
Harshavardhana
ff4e04d942 atomic/fs: use safe package for atomic writes, even in multipart. 2016-04-06 16:05:30 -07:00
Bala FA
dfba4ff397 doc: add multipart documentation about staging files 2016-04-05 19:56:19 -07:00
Harshavardhana
06e3171076 Merge pull request #1290 from balamurugana/devel
Refactor multipart upload
2016-04-05 18:21:46 -07:00
Bala FA
2b3a118636 Merge pull request #1 from harshavardhana/devel
Fix list objects test and remove all the old unnecessary files.
2016-04-06 06:43:43 +05:30
Harshavardhana
8986a6802a Fix ListMultipartUploads 'mc ls -I' now works properly. 2016-04-05 16:39:02 -07:00
Harshavardhana
3fcc60de91 Move the files and rename some functions.
- Rename dir.go as 'fs-multipart-dir.go'
- Move the push/pop to fs-multipart.go and rename them as save/lookup.
- Rename objectInfo instances in fs-multipart as multipartObjInfo.
2016-04-05 12:26:19 -07:00
Harshavardhana
9632c94e7a Fix list objects test and remove all the old unnecessary files.
- Fix tests for new changes.
- Change Golang err as 'e' for the time being, before we bring in
  probe removal change.
- Remove old structs and temporary files.
2016-04-05 12:07:19 -07:00
Bala.FA
083e4e9479 backend/fs: Refactor multipart upload
This patch modifies multipart upload related functions as below

* New multipart upload call creates file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.uploadid

* Put object part call creates file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.PART_NUMBER.MD5SUM_STRING

* Abort multipart call removes all files matching
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.*

* Complete multipart call does
  1. creates a staging file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete.TEMP_NAME
  then renames to
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
  2. rename staging file
  EXPORT_DIR/.minio/BUCKET/PATH/TO/OBJECT/UPLOAD_ID.complete
  to EXPORT_DIR/BUCKET/PATH/TO/OBJECT
2016-04-05 22:22:29 +05:30
Harshavardhana
c69fdf0cf2 listObjects: Cleanup and naming conventions.
- Marker should be escaped outside in handlers.

- Delimiter should be handled outside in handlers.

- Add missing comments and change the function names.

- Handle case of 'maxKeys' when its set to '0', its a valid

  case and should be treated as such.
2016-04-04 19:55:07 -07:00
Krishna Srinivas
85ab1df5a8 listObjects: do not do stat during readdir()
* listObjects: improve response time by not doing stat during readDir() operation.

* listObjects: Add windows support.

* listObjects: Readdir() in batches to conserve memory. Add solaris build.

* listObjects: cleanup code.
2016-04-04 17:27:55 -07:00
Harshavardhana
7623e0f8e8 docker: Fix bug in start.sh arguments. 2016-04-04 11:30:18 -07:00
Anand Babu (AB) Periasamy
9843aa1f7a Merge pull request #1284 from harshavardhana/list
objectAPI: Fix object API interface, remove unnecessary structs.
2016-04-03 21:13:48 -07:00
Harshavardhana
0479d4976b objectAPI: Fix object API interface, remove unnecessary structs.
ObjectAPI changes.
```
ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsInfo, *probe.Error)
ListMultipartUploads(bucket, objectPrefix, keyMarker, uploadIDMarker, delimiter string, maxUploads int) (ListMultipartsInfo, *probe.Error)
ListObjectParts(bucket, object, uploadID string, partNumberMarker, maxParts int) (ListPartsInfo, *probe.Error)
CompleteMultipartUpload(bucket string, object string, uploadID string, parts []completePart) (ObjectInfo, *probe.Error)
```
2016-04-03 15:25:01 -07:00
Anand Babu (AB) Periasamy
12515eabe2 Merge pull request #1280 from harshavardhana/region
signature: No need to validate region for getBucketLocation and listBuckets
2016-04-03 01:01:05 -07:00
Harshavardhana
a6a4e7e297 signature: No need to validate region for getBucketLocation and listBuckets.
This type of check is added for making sure that we can support
custom regions.

ListBuckets and GetBucketLocation are always "us-east-1" rest
should look for the configured region.

Fixes #1278
2016-04-02 18:42:32 -07:00
Anand Babu (AB) Periasamy
2c793a2ea7 Merge pull request #1282 from harshavardhana/remove-old-code
cleanup: Remove old donut/xl code and erasure implementation.
2016-04-02 18:14:21 -07:00
Anand Babu (AB) Periasamy
2bb262cc56 Merge pull request #1279 from harshavardhana/backend
config: Migrate to the new version. Remove backend details.
2016-04-02 17:59:21 -07:00
Harshavardhana
379e0abf03 cleanup: Remove old donut/xl code and erasure implementation.
This is a change to bring in 'klauspost/reedsolomon' library
in #1270 patch.
2016-04-02 17:30:35 -07:00
Harshavardhana
484ba91b08 config: Migrate to the new version. Remove backend details.
Migrate to new config format v4.
```
{
	"version": "4",
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"console": {
			"enable": true,
			"level": "fatal"
		},
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		}
	}
}
```

This patch also updates [minio cli spec](./minio.md)
2016-04-02 17:29:31 -07:00
Harshavardhana
6037fe66e9 minio: Simplify for gosimple tool complaints. 2016-04-02 17:28:54 -07:00
Harshavardhana
ba3a5805c1 vendorize: Add updated ui-assets.go. 2016-04-02 17:27:36 -07:00
Anand Babu (AB) Periasamy
33830bfcae Merge pull request #1273 from harshavardhana/fs-linux
fs: Break fs package to top-level and introduce ObjectAPI interface.
2016-04-01 17:03:23 -07:00
Harshavardhana
efc80343e3 fs: Break fs package to top-level and introduce ObjectAPI interface.
ObjectAPI interface brings in changes needed for XL ObjectAPI layer.

The new interface for any ObjectAPI layer is as below

```
// ObjectAPI interface.
type ObjectAPI interface {
        // Bucket resource API.
        DeleteBucket(bucket string) *probe.Error
        ListBuckets() ([]BucketInfo, *probe.Error)
        MakeBucket(bucket string) *probe.Error
        GetBucketInfo(bucket string) (BucketInfo, *probe.Error)

        // Bucket query API.
        ListObjects(bucket, prefix, marker, delimiter string, maxKeys int) (ListObjectsResult, *probe.Error)
        ListMultipartUploads(bucket string, resources BucketMultipartResourcesMetadata) (BucketMultipartResourcesMetadata, *probe.Error)

        // Object resource API.
        GetObject(bucket, object string, startOffset int64) (io.ReadCloser, *probe.Error)
        GetObjectInfo(bucket, object string) (ObjectInfo, *probe.Error)
        PutObject(bucket string, object string, size int64, data io.Reader, metadata map[string]string) (ObjectInfo, *probe.Error)
        DeleteObject(bucket, object string) *probe.Error

        // Object query API.
        NewMultipartUpload(bucket, object string) (string, *probe.Error)
        PutObjectPart(bucket, object, uploadID string, partID int, size int64, data io.Reader, md5Hex string) (string, *probe.Error)
        ListObjectParts(bucket, object string, resources ObjectResourcesMetadata) (ObjectResourcesMetadata, *probe.Error)
        CompleteMultipartUpload(bucket string, object string, uploadID string, parts []CompletePart) (ObjectInfo, *probe.Error)
        AbortMultipartUpload(bucket, object, uploadID string) *probe.Error
}
```
2016-04-01 15:58:39 -07:00
Harshavardhana
272c5165aa Merge pull request #1272 from krishnasrinivas/get-auth
GetAuth implementation. min/max check for accessKey and secretKey.
2016-04-01 09:38:58 -07:00
Krishna Srinivas
e318925f62 credentials: min/max length check for credentials. 2016-04-01 21:52:39 +05:30
Harshavardhana
2395c42fb5 Merge pull request #1277 from krishnasrinivas/remove-minio-go2
UI-handler: remove minio-go dependancy.
2016-04-01 08:56:16 -07:00
Harshavardhana
9333dc3294 Merge pull request #1204 from hackintoshrao/test-bucket
Test: Changes to TestPutBucket to catch the race
2016-04-01 08:53:34 -07:00
Karthic Rao
30fc970eab Changes to TestPutBucket to catch the race 2016-04-01 15:21:16 +05:30
Krishna Srinivas
331890c4c8 UI-handler: remove minio-go dependancy. 2016-04-01 13:56:32 +05:30
Anand Babu (AB) Periasamy
ae5c65d3c6 Merge pull request #1275 from harshavardhana/signature
error: Signature errors should be returned with APIErrorCode.
2016-03-31 23:37:38 -07:00
Harshavardhana
02ad48466d error: Signature errors should be returned with APIErrorCode.
The reasoning is that we can reply back with wide range of
S3 error responses, which would provide more richer context
to S3 client.

Fixes #1267
2016-03-31 23:28:40 -07:00
Harshavardhana
a84c466a40 Merge pull request #1251 from harshavardhana/release-fixes
release: gz doesn't preserve permissions use tar.gz
2016-03-30 14:34:19 -07:00
Harshavardhana
956142be37 Merge pull request #1271 from krishnasrinivas/set-auth2
UI: implement SetAuth/GenerateAuth handlers for changing credentials.
2016-03-29 09:38:18 -07:00
Krishna Srinivas
5201905ad0 UI: implement SetAuth/GenerateAuth handlers for changing credentials. 2016-03-29 21:08:36 +05:30
Anand Babu (AB) Periasamy
186998ad99 Merge pull request #1266 from harshavardhana/cleanup
routers: Move API and Web routers into their own files.
2016-03-27 14:25:02 -07:00
Harshavardhana
aa8c9bad54 routers: Move API and Web routers into their own files.
This is done to ensure we have a clean way to add new routers such as

  - diskRouter
  - configRouter
  - lockRouter
2016-03-27 13:28:36 -07:00
Harshavardhana
59ee5a547c release: gz doesn't preserve permissions use tar.gz
And fix various other issues with release script.
2016-03-26 23:44:32 -07:00
Harshavardhana
1502e2f29f Merge pull request #1265 from vadmeste/add_fbsd_support
Add simple FreeBSD support, make the minio project compilable
2016-03-26 22:28:07 -07:00
Anand Babu (AB) Periasamy
90a46faf31 Merge pull request #1228 from harshavardhana/signature-cleanup
signature: Move signature outside, use a layered approach for signature verification
2016-03-26 15:46:52 -07:00
Harshavardhana
9dca46e156 signature: Use a layered approach for signature verification.
Signature calculation has now moved out from being a package to
top-level as a layered mechanism.

In case of payload calculation with body, go-routines are initiated
to simultaneously write and calculate shasum. Errors are sent
over the writer so that the lower layer removes the temporary files
properly.
2016-03-26 15:21:05 -07:00
Anis Elleuch
663f24064b Add simple FreeBSD support, make the minio project compilable 2016-03-26 22:39:34 +01:00
Harshavardhana
1b0bc814c4 docker: Fix docker Makefile. 2016-03-24 22:53:13 -07:00
Anand Babu (AB) Periasamy
cd5992c6db Merge pull request #1262 from harshavardhana/docker-file
docker: Fix docker command entry.
2016-03-24 22:44:09 -07:00
Harshavardhana
1ef5ab3c28 docker: Fix docker command entry. 2016-03-24 20:38:36 -07:00
Anand Babu (AB) Periasamy
5bd47861d6 Merge pull request #1261 from harshavardhana/update-message
minio: Server upon start displays a message if update is available.
2016-03-24 20:25:36 -07:00
Harshavardhana
3538c9f598 minio: Server upon start displays a message if update is available.
This code also handles to turn itself off when network is not
available and if request fails. Also prints only when the update
is available.
2016-03-24 20:03:51 -07:00
Anand Babu (AB) Periasamy
24ae5467c8 Merge pull request #1260 from harshavardhana/minio
server: Print a message if not backends are configured.
2016-03-24 17:23:03 -07:00
Harshavardhana
36267eb6e2 server: Print a message if not backends are configured. 2016-03-24 10:47:54 -07:00
Harshavardhana
8255590b3c config/main: set the missing value. 2016-03-24 10:41:42 -07:00
Anand Babu (AB) Periasamy
4f6cf5a6b2 Merge pull request #1123 from harshavardhana/rewrite-v1
config/main: Re-write config files - add to new config v3
2016-03-24 08:52:35 -07:00
Harshavardhana
aaf97ea02c config/main: Re-write config files - add to new config v3
- New config format.

```
{
	"version": "3",
	"address": ":9000",
    "backend": {
          "type": "fs",
          "disk": "/path"
    },
	"credential": {
		"accessKey": "WLGDGYAQYIGI833EV05A",
		"secretKey": "BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF"
	},
	"region": "us-east-1",
	"logger": {
		"file": {
			"enable": false,
			"fileName": "",
			"level": "error"
		},
		"syslog": {
			"enable": false,
			"address": "",
			"level": "debug"
		},
		"console": {
			"enable": true,
			"level": "fatal"
		}
	}
}
```

New command lines in lieu of supporting XL.

Minio initialize filesystem backend.
~~~
$ minio init fs <path>
~~~

Minio initialize XL backend.
~~~
$ minio init xl <url1>...<url16>
~~~

For 'fs' backend it starts the server.
~~~
$ minio server
~~~

For 'xl' backend it waits for servers to join.
~~~
$ minio server
... [PROGRESS BAR] of servers connecting
~~~

Now on other servers execute 'join' and they connect.
~~~
....
minio join <url1> -- from <url2> && minio server
minio join <url1> -- from <url3> && minio server
...
...
minio join <url1> -- from <url16> && minio server
~~~
2016-03-23 19:16:09 -07:00
Harshavardhana
85e50f2bb9 Merge pull request #1258 from hackintoshrao/list-object-prefix-response
api: GetObjectInfo fix
2016-03-23 11:09:22 -07:00
Karthic Rao
c8570edaab Issue of 'mc ls' when prefix is a directory fixed, tests added for GetObjectInfo 2016-03-23 19:42:04 +05:30
Anand Babu (AB) Periasamy
f1161d830f Merge pull request #1257 from harshavardhana/content-md5
routers: Fix order of PostPolicyHandlers and headers.
2016-03-22 20:49:21 -07:00
Anand Babu (AB) Periasamy
4aa098ede9 Merge pull request #1256 from harshavardhana/resources
bucketpolicy: checkBucketPolicy should keep resources in map.
2016-03-22 20:47:48 -07:00
Harshavardhana
76bda0d8f1 routers: Fix order of PostPolicyHandlers and headers. 2016-03-22 17:54:44 -07:00
Harshavardhana
996d2e2a10 bucketpolicy: checkBucketPolicy should keep resources in map.
This is done to make sure to avoid appending duplicates for
resources for each actions.
2016-03-22 17:04:39 -07:00
Harshavardhana
2edf32adfa Merge pull request #1253 from koolhead17/patch-5
Update README.md
2016-03-22 17:01:59 -07:00
Harshavardhana
e3a3283883 Merge pull request #1255 from hackintoshrao/list-object-prefix-response
api: ListObject - Changing to empty response when prefixDir doesn't exist
2016-03-22 17:01:48 -07:00
Karthic Rao
7be79b507b Changing to empty response when prefixDir doesn't exist 2016-03-23 04:46:10 +05:30
Harshavardhana
600a932acb Merge pull request #1254 from hackintoshrao/formatting-fix
Formating: Formating issues fixed
2016-03-22 04:34:57 -07:00
Karthic Rao
ff41c050d5 Formatting issues fixed. 2016-03-22 15:55:29 +05:30
koolhead17
7f993bb5e6 Update README.md
fixed missing spelling,
2016-03-22 13:37:04 +05:30
Harshavardhana
7a97622fed Merge pull request #1252 from koolhead17/patch-4
docs: Add more s3cmd commands
2016-03-22 01:04:35 -07:00
koolhead17
da691dc100 Update README.md
Added more s3cmd commands associated & known to work well with Minio server
2016-03-22 12:39:44 +05:30
Harshavardhana
e2c515b334 Merge pull request #1245 from hackintoshrao/fs-bucket-tests
api: ListObject - tests, benchmark, optimization
2016-03-21 21:51:48 -07:00
Karthic Rao
b55922effe Fix for Istruncated set to true under certain conditions.
Optimizing List Objects by using binary sort to discard entries in cases
where prefix or marker is set.

Adding test coverage to ListObjects.

Adding benchmark to ListObjects.
2016-03-22 10:09:16 +05:30
Anand Babu (AB) Periasamy
407316c631 Merge pull request #1249 from harshavardhana/remove-deadcode
main: Remove all the dead/unused code.
2016-03-21 07:55:07 -07:00
Harshavardhana
902aa05021 main: Remove all the dead/unused code.
This patch removes some dead and unused code.
2016-03-21 01:12:29 -07:00
Anand Babu (AB) Periasamy
d95aac4b36 Merge pull request #1246 from harshavardhana/list-response
api: ListMultipartUploads and ListParts responded more entries.
2016-03-20 00:07:06 -07:00
Harshavardhana
fc72a0362f api: ListMultipartUploads and ListParts responded more entries.
Issue is empty entries were added since allocating an array was
followed by an append. Keep the index and copy the right entries
precisely.

Fixes an issue reported at - https://github.com/minio/mc/issues/1642
2016-03-19 23:52:40 -07:00
Anand Babu (AB) Periasamy
1900d16d22 Merge pull request #1244 from harshavardhana/zip-compress
buildscripts: compress release binaries.
2016-03-18 23:37:52 -07:00
Harshavardhana
41cba3a457 buildscripts: compress release binaries.
Fix update command as well to show compressed files in updates.
2016-03-18 23:30:54 -07:00
Harshavardhana
a90faf5996 Merge pull request #1216 from sreeram-boyapati/choose_arch
buildscripts: Enable user to choose an arch to build
2016-03-18 12:13:03 -07:00
Harshavardhana
e7e2fae156 Merge pull request #1238 from hackintoshrao/maxkeys-zero-fix
[GH-1237] api: Handling maxKeys=0 case with a empty response
2016-03-18 02:57:40 -07:00
Karthic Rao
99af0444b7 Handling maxKeys=0 case with a empty response 2016-03-18 15:16:30 +05:30
Harshavardhana
2357e00317 Fix s3cmd config 2016-03-18 02:07:17 -07:00
Anand Babu (AB) Periasamy
27c75d8be4 Merge pull request #1241 from harshavardhana/print
[GH-1240] main: Print keys after init and full server initialization.
2016-03-17 23:42:46 -07:00
Harshavardhana
72d364cbf2 [GH-1240] main: Print keys after init and full server initialization.
Setting MINIO_ACCESS_KEY and MINIO_SECRET_KEY re-writes the values
in config properly, but the init message is not updated.

Fix it by delay printing keys until everything is properly
initialized.
2016-03-17 20:01:52 -07:00
Harshavardhana
a729d02c31 Merge pull request #1227 from krishnasrinivas/docker-md
docker: simplify docker run instructions.
2016-03-16 18:38:37 -07:00
Harshavardhana
c80c06680d Merge pull request #1235 from krishnasrinivas/handle-eaddrinuse
startup: do not fail for non-EADDRINUSE errors. Fixes #1234
2016-03-16 18:38:27 -07:00
Harshavardhana
facd5a9ffd Merge pull request #1239 from awwalker/verify-headers
api: CopyObject - verify the additional headers before starting to write.
2016-03-16 18:37:56 -07:00
awwalker
34f2c5bcdf verify before writing
merge

verify headers before writing
2016-03-16 18:03:23 -07:00
Krishna Srinivas
fd943c704d docker: simplify docker run instructions. 2016-03-17 01:39:49 +05:30
Krishna Srinivas
e5b411caf4 startup: do not fail for non-EADDRINUSE errors. Fixes #1234 2016-03-17 01:36:08 +05:30
Anand Babu (AB) Periasamy
c784707ed8 Merge pull request #1233 from harshavardhana/bucket-policy
bucketpolicy: Improve bucket policy validation, avoid nested rules.
2016-03-15 18:57:02 -07:00
Harshavardhana
88714e7c8e bucketpolicy: Improve bucket policy validation, avoid nested rules.
Bucket policy validation is more stricter now, to avoid nested
rules. The reason to do this is keep the rules simpler and more
meaningful avoiding conflicts.

This patch implements stricter checks.

Example policy to be generally avoided.
```
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Action": [
				"s3:GetObject",
				"s3:DeleteObject"
			],
			"Effect": "Allow",
			"Principal": {
				"AWS": [
					"*"
				]
			},
			"Resource": [
				"arn:aws:s3:::jarjarbing/*"
			]
		},
		{
			"Action": [
				"s3:GetObject",
				"s3:DeleteObject"
			],
			"Effect": "Deny",
			"Principal": {
				"AWS": [
					"*"
				]
			},
			"Resource": [
				"arn:aws:s3:::jarjarbing/restic/key/*"
			]
		}
	]
}
```
2016-03-15 17:50:23 -07:00
Harshavardhana
2e3e164f16 Update Docker.md 2016-03-15 10:01:58 -07:00
Harshavardhana
15b5f4d7a3 Update Caddy.md 2016-03-14 20:51:16 -07:00
Harshavardhana
83f9a50137 Merge pull request #1229 from brendanashworth/improve-fs-getobject
pkg/fs: optimize GetObject, add benchmark
2016-03-13 14:03:45 -07:00
Brendan Ashworth
583e4ecff6 pkg/fs: optimize GetObject syscalls for common case
In the common case, GetObject is called on a bucket that exists and an
object that exists and is not a directory. It should be optimized for
this case, thus error-related syscalls are pushed back until they are
necessary.

This should not impact performance negatively in the uncommon case, and
instead drops two otherwise unnecessary os.Stat's in the common case.

The race conditions around a proper error being returned were present
beforehand.

It also renames 'err' to 'e'.
2016-03-13 13:56:33 -07:00
Brendan Ashworth
b2257682e4 pkg/fs: add benchmark for GetObject
This commit adds a benchmark for GetObject. It doesn't leverage the I/O
as much because it uses short text for data, just 58 chars.
2016-03-13 11:13:06 -07:00
Harshavardhana
09789de586 Merge branch 'awwalker-new-copy-headers' 2016-03-12 10:56:11 -08:00
awwalker
9a5e3299fc api/object: Add CopyObject to support match/modified copy headers
Adds support for the following request headers:

- x-amz-copy-source-if-match
- x-amz-copy-source-if-none-match
- x-amz-copy-source-if-unmodified-since
- x-amz-copy-source-if-modified-since

Fixes #1176
2016-03-12 10:54:23 -08:00
Harshavardhana
2c09df24ee Merge pull request #1225 from hackintoshrao/fs-bucket-tests
tests: Add tests for ListBuckets
2016-03-12 01:26:56 -08:00
Karthic Rao
53a76439a2 test for GetBucketInfo 2016-03-12 14:31:30 +05:30
Anand Babu (AB) Periasamy
6b0af08885 Merge pull request #1224 from harshavardhana/simplify
cleanup: Remove unecessary packages and tests. Simplify.
2016-03-11 22:58:43 -08:00
Harshavardhana
5282a79eda cleanup: Remove unecessary packages and tests. Simplify. 2016-03-11 19:53:55 -08:00
Anand Babu (AB) Periasamy
3d8b9afa8f Merge pull request #1222 from harshavardhana/cleanup-fix
cleanup: Rename ObjectMetadata as ObjectInfo.
2016-03-11 17:14:22 -08:00
Harshavardhana
52751d81cb cleanup: Rename ObjectMetadata as ObjectInfo.
Fixes #1215
2016-03-11 16:58:08 -08:00
Harshavardhana
c81f4b0228 Merge pull request #1218 from hackintoshrao/better-fs-util-test
Test: Better structuring of fs-utils test
2016-03-11 06:54:43 -08:00
Karthic Rao
ec8c1d4ef6 Better structuring of fs-utils test 2016-03-11 19:19:47 +05:30
Sreeram Boyapati
62bd44f873 buildscripts: Enable user to choose an arch to build
- Building minio for all architectures takes a lot of time.
    Choose the one user needs
2016-03-11 10:17:56 +05:30
Anand Babu (AB) Periasamy
b5c77b641d Merge pull request #1209 from harshavardhana/err-naming
error: Add proper prefixes for s3Error codes.
2016-03-10 18:59:16 -08:00
Harshavardhana
fdf3d64793 error: Add proper prefixes for s3Error codes.
This patch adds 'Err' prefix for all standard API
error codes and also adds a proper type for them.
2016-03-10 18:38:46 -08:00
Anand Babu (AB) Periasamy
373d335d94 Merge pull request #1214 from brendanashworth/improve-listbuckets
ListBuckets test & improvement, IsValid{Bucket,Object}Name fix, test, docs
2016-03-10 18:20:18 -08:00
Anand Babu (AB) Periasamy
b16025abf4 Merge pull request #1213 from harshavardhana/presigned
auth: Detect anonymous as the last resort.
2016-03-10 18:11:43 -08:00
Harshavardhana
166ef09c3d auth: Detect anonymous as the last resort. 2016-03-10 17:55:36 -08:00
Harshavardhana
e54aa10201 Merge pull request #1212 from harshavardhana/venodr
vendor: Update ui-assets with new changes and release.
2016-03-10 17:55:25 -08:00
Harshavardhana
5606232567 vendor: Update ui-assets with new changes and release. 2016-03-10 17:36:32 -08:00
Harshavardhana
9352cb87c6 Merge pull request #1211 from harshavardhana/vendorize
vendor: Add minio-go vendor updates.
2016-03-10 15:50:14 -08:00
Harshavardhana
e781959d5b vendor: Add minio-go vendor updates. 2016-03-10 14:33:15 -08:00
Harshavardhana
af295f3600 Merge pull request #1186 from balamurugana/devel
api: refactor list object handling in fs backend
2016-03-10 13:43:49 -08:00
Bala.FA
c70bc2209e api: refactor list object handling in fs backend
When list object is invoked, it creates a goroutine if not available
for given parameters else uses existing goroutine.  These goroutines
are alive for 15 seconds for further continuation list object request
else they exit.

Fixes #1076
2016-03-11 02:20:51 +05:30
Harshavardhana
5cb546d288 Merge pull request #1210 from krishnasrinivas/port-check-fix
startup: specify the network - tcp4/tcp6 for ListenTCP()
2016-03-10 08:54:02 -08:00
Krishna Srinivas
010e775b17 startup: specify the network - tcp4/tcp6 for ListenTCP() 2016-03-10 17:17:47 +05:30
Harshavardhana
2cba605514 Merge pull request #1206 from fwessels/notice-typo
Fix typo
2016-03-09 09:25:24 -08:00
Harshavardhana
d740d3aaae Merge pull request #1208 from krishnasrinivas/port-check
startup: do not start minio server if port is not free. Fixes #1207
2016-03-09 09:25:05 -08:00
Anand Babu (AB) Periasamy
5ac4afa4d1 Merge pull request #1080 from harshavardhana/bucket-policy
accessPolicy: Implement Put, Get, Delete access policy.
2016-03-09 07:11:39 -08:00
Krishna Srinivas
ea7ea427ca startup: do not start minio server if port is not free. Fixes #1207 2016-03-09 19:39:40 +05:30
frankw
027f7efbdb Fix typo 2016-03-09 13:55:58 +01:00
Harshavardhana
d5057b3c51 accessPolicy: Implement Put, Get, Delete access policy.
This patch implements Get,Put,Delete bucket policies

Supporting - http://docs.aws.amazon.com/AmazonS3/latest/dev/access-policy-language-overview.html

Currently supports following actions.

   "*":                             true,
   "s3:*":                          true,
   "s3:GetObject":                  true,
   "s3:ListBucket":                 true,
   "s3:PutObject":                  true,
   "s3:CreateBucket":               true,
   "s3:GetBucketLocation":          true,
   "s3:DeleteBucket":               true,
   "s3:DeleteObject":               true,
   "s3:AbortMultipartUpload":       true,
   "s3:ListBucketMultipartUploads": true,
   "s3:ListMultipartUploadParts":   true,

following conditions for "StringEquals" and "StringNotEquals"

   "s3:prefix", "s3:max-keys"
2016-03-08 17:44:50 -08:00
Harshavardhana
846410c563 Merge pull request #1205 from harshavardhana/patch-3
Fix docker documentation.
2016-03-08 11:45:45 -08:00
Harshavardhana
f268762ec7 Fix docker documentation. 2016-03-08 11:23:24 -08:00
Atul Jha
41739e0913 Fix doc as per community member suggestion.
Added suggestion mentioned by our community member - fixes #1116
2016-03-08 10:42:56 -08:00
Brendan Ashworth
cd3eb63c4a pkg/fs: test, document, and fix IsValid{Bucket,Object}Name
This commit improves the docs for both functions (more Go-like) and
drops an unnecessary condition in IsValidBucketName. This also drops a
condition in IsValidObjectName where "" (empty string) was a valid
object name. This has been fixed and will no longer return true.

This commit also adds tests for both functions, including a regression
test for the bug fix.
2016-03-07 19:59:24 -08:00
Brendan Ashworth
a5d0bef4e2 pkg/fs: test, bench, and drop unnecessary check in ListBuckets
There is now a simple test and a benchmark for ListBuckets. I also
dropped an unnecessary check that was simply repeated from above,
guaranteed to be true.
2016-03-07 19:58:33 -08:00
Harshavardhana
114f9de5eb Merge pull request #1201 from harshavardhana/time
handlers: Cleanup time handler helpers.
2016-03-07 12:05:26 -08:00
Harshavardhana
761cb2c740 handlers: Cleanup time handlers helpers. 2016-03-07 10:47:45 -08:00
Harshavardhana
59fbb6d081 Merge pull request #1199 from brendanashworth/improvements-fs
pkg/fs improvements
2016-03-07 00:31:57 -08:00
Brendan Ashworth
fab45aae40 pkg/fs: add bucket test and benchmarks
Lots of useful benchmarks and a simple test addition!
2016-03-07 00:07:11 -08:00
Brendan Ashworth
7399d8ceaa pkg/fs: skip unnecessary os.Stat system call 2016-03-07 00:07:11 -08:00
Brendan Ashworth
0a0451a0fb pkg/fs: DRY SetBucketMetadata
It had a lot of code that was the same as GetBucketMetadata, so instead
call GBM from SBM so as to reduce doing the same thing in two different
spots. Theoretically this will induce a small overhead as now at least
two calls of denormalizeBucket are made, although this shouldn't be
noticeable.
2016-03-07 00:07:11 -08:00
Brendan Ashworth
294ea814bf pkg/fs: for locks, prefer defer and read-only ops
This commit prefers the use of 'defer' for fs.Unlock (and fs.RUnlock)
because it is more idiomatic Go and reduces repetition in the code,
lending to a cleaner code base.

It also switches a few uses of the lock to read-only locks, which should
improve performance of those functions dramatically in certain contexts.
2016-03-07 00:07:11 -08:00
Anand Babu (AB) Periasamy
d54a8a9c07 Merge pull request #1195 from harshavardhana/delete-objects
api: Implement multiple objects Delete api - fixes #956
2016-03-06 18:52:39 -08:00
Harshavardhana
aed62788d9 api: Implement multiple objects Delete api - fixes #956
This API takes input XML input in following form.

```
<?xml version="1.0" encoding="UTF-8"?>
<Delete>
    <Quiet>true</Quiet>
    <Object>
         <Key>Key</Key>
    </Object>
    <Object>
         <Key>Key</Key>
    </Object>
    ...
</Delete>
```

and responds the list of successful deletes, list of errors
for all the deleted objects.

```
<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Deleted>
    <Key>sample1.txt</Key>
  </Deleted>
  <Error>
    <Key>sample2.txt</Key>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
  </Error>
</DeleteResult>
```
2016-03-06 18:31:50 -08:00
Harshavardhana
6f842124ad Merge pull request #1197 from brendanashworth/improve-DRY-1
api: DRY code and add new test
2016-03-06 16:30:48 -08:00
Brendan Ashworth
adf74ffdb0 api: DRY code and add new test
This commit makes code cleaner and reduces the repetitions in the code
base. Specifically, it reduces the clutter in setObjectHeaders. It also
merges encodeSuccessResponse and encodeErrorResponse together because
they served no purpose differently. Finally, it adds a simple test for
generateRequestID.
2016-03-06 13:26:27 -08:00
Harshavardhana
e883583804 Merge pull request #1192 from harshavardhana/color
console: Fix console color printing on windows.
2016-03-04 13:50:49 -08:00
Harshavardhana
164dfe2ec9 console: Fix console color printing on windows.
Print colored accessKeyID and secretAccessKey are unreadable on windows
on command prompts and powershell.

Use the console package from minio client.
2016-03-04 10:07:19 -08:00
Harshavardhana
91800cff53 Merge pull request #1189 from krishnasrinivas/assets-gzip-encoding
browser-assets: serve asset files with compression.
2016-03-04 01:12:11 -08:00
Krishna Srinivas
44b2037667 browser-assets: serve asset files with compression. 2016-03-04 07:47:26 +05:30
Harshavardhana
8bf882e5c0 Merge pull request #1188 from harshavardhana/vendorize
browser: vendorize to new browser update
2016-03-03 18:04:09 -08:00
Harshavardhana
95d3ecb9ce browser: vendorize to new browser update 2016-03-03 17:46:09 -08:00
Harshavardhana
97472e76b0 Merge pull request #1187 from harshavardhana/support-newfiles
routers: Support special asset files.
2016-03-03 17:25:25 -08:00
Harshavardhana
df0bce374c routers: Support special asset files.
- loader.css
 - logo.svg
 - {firefox,chrome,safari}.png
2016-03-03 17:13:15 -08:00
Harshavardhana
356b889e66 Merge pull request #1184 from harshavardhana/multipart
multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
2016-03-02 18:42:24 -08:00
Harshavardhana
f111997184 multipart: remove proper MD5, rather create MD5 based on parts to be s3 compatible.
This increases the performance phenominally.
2016-03-02 14:20:49 -08:00
Harshavardhana
5f2cfdfbe2 Merge pull request #1185 from harshavardhana/fix-signature
signature: Fix signature handling of parallel requests.
2016-03-02 14:20:18 -08:00
Harshavardhana
17d145df3a signature: Fix signature handling of parallel requests.
Signature struct should be immutable, this fixes an issue
with AWS cli not being able to do multipart put operations.
2016-03-02 11:49:50 -08:00
GarimaKapoor
b37fbabe7f Update README.md 2016-03-02 11:07:23 -08:00
Harshavardhana
d7767d6431 Merge pull request #1181 from hackintoshrao/vet-shadow-fix
build: Removing error initialization to solve variable shadowing
2016-03-01 21:26:15 -08:00
Karthic Rao
6651f5b368 go vet shadow error patch 2016-03-02 09:55:00 +05:30
Harshavardhana
dc622674e1 Merge pull request #1178 from harshavardhana/list-objects
list: Fix handling of maxKeys and prefixes.
2016-03-01 17:55:19 -08:00
Harshavardhana
c7021f6a95 list: Fix handling of maxKeys and prefixes.
This fixes a problem of requeuing the same request
and also fixes a major problem of sending truncated
for full key prefixes.

Fixes #1177
2016-03-01 17:34:31 -08:00
Harshavardhana
53ca192fe7 Merge pull request #1175 from harshavardhana/get-object
api: Implement support for additional request headers.
2016-02-28 19:42:40 -08:00
Harshavardhana
ee1b86e517 api: Implement support for additional request headers.
Now GetObject and HeadObject both support

  - If-Modified-Since, If-Unmodified-Since
  - If-Match, If-None-Match

request headers.

These headers are used to further handle the responses for GetObject
and HeadObject API.

Fixes #1098
2016-02-28 19:34:20 -08:00
Harshavardhana
0b2e449727 Merge pull request #1174 from harshavardhana/copy-object
api: Implement CopyObject s3 API, doing server side copy.
2016-02-27 20:44:56 -08:00
Harshavardhana
3ff8a1b719 api: Implement CopyObject s3 API, doing server side copy.
Fixes #1172
2016-02-27 19:51:59 -08:00
Harshavardhana
2520298734 Merge pull request #1170 from krishnasrinivas/caching
caching: disable caching of index.html and enable caching for other UI asset files.
2016-02-27 00:02:51 -08:00
Krishna Srinivas
af7170675d caching: disable caching of index.html and enable caching for other UI asset files. 2016-02-27 12:22:26 +05:30
Harshavardhana
7780a1ce4c Merge pull request #1171 from harshavardhana/disable-multi
api: Return NotImplemented for MultiDelete and CopyObject APIs
2016-02-26 16:34:37 -08:00
Harshavardhana
ae6e774377 api: Return NotImplemented for MultiDelete and CopyObject APIs 2016-02-26 15:57:30 -08:00
Harshavardhana
2ec211e52a Merge pull request #1169 from minio/harshavardhana-patch-1
Update README.md
2016-02-24 20:30:29 -08:00
Harshavardhana
9122f06307 Update README.md 2016-02-24 19:57:11 -08:00
Harshavardhana
04d6408c31 Merge pull request #1167 from harshavardhana/fix-release-tag
build: Fix release tag.
2016-02-23 17:16:26 -08:00
Harshavardhana
024c00addd build: Fix release tag. 2016-02-23 16:56:41 -08:00
Harshavardhana
2c26b98242 Merge pull request #1166 from harshavardhana/release
build: Add release builds, now generated with 'make release'
2016-02-23 16:02:51 -08:00
Harshavardhana
223245cc45 build: Add release builds, now generated with 'make release'
Currently supported platforms are

    - linux{amd64,arm,386}
    - winows{amd64,386}
    - darwin{amd64}
2016-02-23 15:14:02 -08:00
Harshavardhana
781540081d vendor: Update to new upstream changes from fatih/color
Brings in changes like support for Solaris/FreeBSD.
2016-02-23 15:01:17 -08:00
Harshavardhana
acb44e294c Merge pull request #1140 from harshavardhana/go1.6
build/vet: Fix all the shadowing reports with go1.6
2016-02-23 14:50:46 -08:00
Harshavardhana
408aa72146 build/vet: Fix all the shadowing reports with go1.6
Golang 1.6 is default version for the build now.

Additionally set 'GODEBUG=cgocheck=0' for now, until
we fix the erasure coding package.

Readmore here https://tip.golang.org/doc/go1.6#cgo
2016-02-23 14:34:39 -08:00
Harshavardhana
04424ae9c2 Merge pull request #1163 from harshavardhana/minio-browser
web: Removing dependency for gpg and downloading assets.
2016-02-23 14:16:25 -08:00
Harshavardhana
2181003609 web: Removing dependency for gpg and downloading assets.
Assets are vendorized from now on and updated for each release.
2016-02-23 13:32:12 -08:00
Harshavardhana
997141d031 Merge pull request #1161 from krishnasrinivas/feb-23
UI: serve index.html if the requested file is not found in the assets bundle.
2016-02-23 12:14:52 -08:00
Krishna Srinivas
e509bcb2b9 UI: serve index.html if the requested file is not found in the assets bundle. 2016-02-24 00:36:30 +05:30
Anand Babu (AB) Periasamy
07da31f8b8 Merge pull request #1150 from harshavardhana/signature
signV4: Move pkg/signature to pkg/s3/signature4
2016-02-23 12:39:28 +05:30
Harshavardhana
653ceee9ee signV4: Move pkg/signature to pkg/s3/signature4
Cleanup and move this to relevant path.
2016-02-22 22:47:09 -08:00
Anand Babu (AB) Periasamy
b31dac9162 Merge pull request #1144 from harshavardhana/definitions
cleanup: Remove definitions and move them to its relative places acco…
2016-02-23 01:50:17 +05:30
Harshavardhana
31c941d320 Merge branch 'osallou-feature_credentials_envvars' 2016-02-22 10:49:14 -08:00
Olivier Sallou
678585c513 use environment variables to set and override access and secret keys at server startup 2016-02-22 10:49:02 -08:00
Harshavardhana
800b19d8e5 cleanup: Remove definitions and move them to its relative places accordingly
- Move fs-definitions.go and break them into fs-datatypes.go, fs-bucket-acl.go
  and fs-utils.go
- Move api-definitions.go to api-response.go, where they should be.
- Move web-definitions to its related handlers.
2016-02-22 10:41:27 -08:00
Harshavardhana
bd6850e79f Merge pull request #1155 from harshavardhana/verify-yasm
build: Verify yasm version and complain - fixes #1154
2016-02-22 03:26:03 -08:00
Harshavardhana
18fd0a0f81 build: Verify yasm version and complain - fixes #1154 2016-02-22 03:17:17 -08:00
Harshavardhana
8824b18c29 Merge pull request #1151 from harshavardhana/bucket-fix
router: Fix "/minio" router for web.
2016-02-21 23:05:43 -08:00
Harshavardhana
5da1972d1f router: Fix "/minio" router for web. 2016-02-21 22:59:01 -08:00
Harshavardhana
d3966d1dde Merge pull request #1145 from harshavardhana/bucket
web: Handle private bucket match from prefix to exact match.
2016-02-21 12:14:09 -08:00
Harshavardhana
393636f74b Merge pull request #1149 from alexex/master
Alpine & CA-Certs for SSL support
2016-02-20 02:24:03 -08:00
Harshavardhana
91a7b13529 web: Handle private bucket match from prefix to exact match.
Filter out 'privateBucket' if any from listBuckets output.
2016-02-20 02:22:56 -08:00
Harshavardhana
185d29a899 Merge pull request #1148 from harshavardhana/fix-bug
web: Add targetProto for putObjectURL,getObjectURL SSL requests.
2016-02-20 02:15:56 -08:00
Alexander Jung-Loddenkemper
342f8fbe5d Alpine & CA-Certs for SSL support
Switch to alpine as a base image and install the ca certifcates to enable the minio docker container to run behind a SSL proxy, which will fix #1146
2016-02-20 11:15:19 +01:00
Harshavardhana
4db136c19c web: Add targetProto for putObjectURL,getObjectURL SSL requests.
Currently the default request was based on the 'minio server'
SSL configuration. But inside a proxy this is invalid browser
needs to send which protocol it wishes the PresignedURL
should be generated for.
2016-02-20 01:59:18 -08:00
Harshavardhana
a18620fa86 Merge pull request #1141 from harshavardhana/target-host
web: PresignedGet/Put targetHost should always have a valid value.
2016-02-19 15:53:20 -08:00
Harshavardhana
d4c91426a7 web: PresignedGet/Put targetHost should always have a valid value.
PresignedURLs should always be generated for the targetHost, so that
we can avoid signature issues.
2016-02-19 15:31:38 -08:00
Harshavardhana
51e611a380 Merge pull request #1139 from harshavardhana/docker
docker: Make sure that we properly check for containers.
2016-02-18 13:59:22 -08:00
Harshavardhana
354229732b docker: Make sure that we properly check for containers. 2016-02-18 13:39:44 -08:00
Harshavardhana
1c33926239 Merge pull request #1136 from harshavardhana/add-proxy
presigned: Fix a bug in presigned request verification.
2016-02-18 02:46:04 -08:00
Harshavardhana
91a092792a presigned: Fix a bug in presigned request verification.
Additionally add Docker proxy configuration.
2016-02-18 02:23:12 -08:00
Harshavardhana
d561f0cc0b Merge pull request #1135 from abperiasamy/vendor-update-go-homedir
vendor update for go-homedir
2016-02-18 00:14:58 -08:00
Anand Babu (AB) Periasamy
f53e9dd1b8 vendor update for go-homedir 2016-02-18 13:02:41 +05:30
Harshavardhana
2a6bc604db Merge pull request #1132 from harshavardhana/merge-ports
web/rpc: Merge ports with API server.
2016-02-17 21:22:48 -08:00
Harshavardhana
dd9aaa855c web/rpc: Merge ports with API server.
Fixes #1081 and #1130
2016-02-17 20:28:15 -08:00
Harshavardhana
5a9333a67b signature: Rewrite signature handling and move it into a library. 2016-02-16 17:28:16 -08:00
Harshavardhana
b531bb31bb Merge pull request #1129 from harshavardhana/cpu-klauspost
cpu: Remove pkg/cpu in favor of better klauspost/cpuid.
2016-02-15 14:06:05 -08:00
Harshavardhana
9e10ee7e47 cpu: Remove pkg/cpu in favor of better klauspost/cpuid.
Fixes #1128
2016-02-15 13:50:33 -08:00
Harshavardhana
a173313bc2 Merge pull request #1127 from abperiasamy/master
rewrite minio runtime checks
2016-02-15 10:39:01 -08:00
Anand Babu (AB) Periasamy
bbca70e13b rewrite minio runtime checks 2016-02-15 17:56:56 +05:30
Harshavardhana
e73301944a Merge pull request #1126 from harshavardhana/fix-multipart
fs/multipart: Handle un-ordered creation of multiparts.
2016-02-14 00:49:07 -08:00
Harshavardhana
fbab7128d5 fs/multipart: Handle un-ordered creation of multiparts.
Fixes #1125
2016-02-14 00:39:15 -08:00
Harshavardhana
d38bbf3562 Merge pull request #1124 from harshavardhana/json-rpc-fix
rpc: Fix json rpc to handle array and object request params.
2016-02-13 19:15:52 -08:00
Harshavardhana
e59ceba51b rpc: Fix json rpc to handle array and object request params.
rpc/v2/json2 code has a bug where it treats all jsonrpc 2.0
request params like an 'object'. In accordance with the spec
it could be both 'object' or an 'array'.

Handle both cases.
2016-02-13 19:01:36 -08:00
Harshavardhana
40e8893fb9 Merge pull request #1122 from harshavardhana/osx-build
build: Simplify and build only with Makefiles.
2016-02-13 01:27:15 -08:00
Harshavardhana
ebdbe2db44 build: Simplify and build only with Makefiles.
Configure is not portable.
2016-02-13 01:19:13 -08:00
Harshavardhana
3fd637c1f4 Merge branch 'json-rpc' 2016-02-12 20:32:47 -08:00
Krishna Srinivas
56ffe79f25 jsonrpc: return json2.Error from the json-rpc layer to the client 2016-02-12 20:30:02 -08:00
Krishna Srinivas
6ad39cb386 WebUI: move from rpc/v2/json to rpc/v2/json2 which has better error response structure. 2016-02-12 20:29:56 -08:00
Harshavardhana
f98675660b Merge pull request #1119 from harshavardhana/minio
xl: Moved to minio/minio
2016-02-11 15:54:02 -08:00
Harshavardhana
62f6ffb6db xl: Moved to minio/minio - fixes #1112 2016-02-11 15:43:36 -08:00
Harshavardhana
f0e2edd632 Merge pull request #1120 from krishnasrinivas/version-check
UI: implement rpc call to return UI version
2016-02-11 11:07:57 -08:00
Krishna Srinivas
5e32dec4fb UI: implement rpc call to return UI version 2016-02-12 00:26:27 +05:30
Harshavardhana
33bd97d581 Merge pull request #1118 from harshavardhana/make-file
build: Change UI assets location.
2016-02-10 16:21:58 -08:00
Harshavardhana
70bbf4c8ec build: Change UI assets location. 2016-02-10 16:10:33 -08:00
Harshavardhana
785a1f0eb0 Merge pull request #1117 from harshavardhana/devel
pkg/ioutils: remove usage of os.Lstat() in FTW()
2016-02-10 13:52:16 -08:00
Harshavardhana
6e9d73426b pkg/ioutils: True should be true 2016-02-10 13:33:36 -08:00
Bala.FA
5e4b13f4bd remove unused functions 2016-02-10 13:32:53 -08:00
Bala.FA
255505a83b pkg/ioutils: remove usage of os.Lstat() in FTW()
As os.Readdir() is used get file entries where statinfo is already
present.  This patch fixes to use statinfo provided by os.Readdir().
2016-02-10 13:32:53 -08:00
Harshavardhana
31dbdb1787 Merge pull request #1113 from harshavardhana/list-objects-optimization
listObjects: list objects minor optimization.
2016-02-10 10:33:37 -08:00
Harshavardhana
7c38430710 Merge pull request #1115 from krishnasrinivas/return-error-type
jsonrpc: WrapError() makes jsonrpc return unnecessary details in the error message.
2016-02-10 10:18:12 -08:00
Krishna Srinivas
318265ecaf jsonrpc: WrapError() makes jsonrpc return unnecessary details in the error message. 2016-02-10 21:22:37 +05:30
Harshavardhana
98ee5fcf55 build: Add spelling checks and check if curl is installed. 2016-02-10 00:18:05 -08:00
Harshavardhana
9b29af8bbe listObjects: list objects minor optimization.
Minor optimization.

- Add 1000 entries buffered channel for walkerCh.
- Reset marker after the lexical order has reached and
  compare only if the marker is set.
2016-02-09 21:45:19 -08:00
Harshavardhana
4ef01bc225 Merge pull request #1109 from krishnasrinivas/getuiversion
rpc: Add rpc uiVersion string
2016-02-09 10:29:28 -08:00
Krishna Srinivas
bf6078df13 webrpc: Implement GetUIVersion json-rpc API. 2016-02-09 18:09:32 +05:30
Harshavardhana
4e328d7b2c Merge pull request #1106 from harshavardhana/support
pkg/user: Support 32bit darwin.
2016-02-08 01:59:02 -08:00
Harshavardhana
42fcb27308 pkg/user: Support 32bit darwin in user package. 2016-02-08 01:34:25 -08:00
1647 changed files with 388882 additions and 53138 deletions

33
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,33 @@
<!--- Provide a general summary of the issue in the Title above -->
## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->
<!--- If you're suggesting a change/improvement, tell us how it should work -->
## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->
<!--- If suggesting a change/improvement, explain the difference from current behavior -->
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
<!--- or ideas how to implement the addition or change -->
## Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
1.
2.
3.
4.
## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used (`minio version`):
* Environment name and version (e.g. nginx 1.9.1):
* Server type and version:
* Operating System and version (`uname -a`):
* Link to your project:

28
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,28 @@
<!--- Provide a general summary of your changes in the Title above -->
## Description
<!--- Describe your changes in detail -->
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here. -->
## How Has This Been Tested?
<!--- Please describe in detail how you tested your changes. -->
<!--- Include details of your testing environment, and the tests you ran to -->
<!--- see how your change affects other areas of the code, etc. -->
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.

9
.gitignore vendored
View File

@@ -9,5 +9,10 @@ site/
/.idea/
/Minio.iml
**/access.log
ui-assets.go
ui-assets.asc
build
vendor/**/*.js
vendor/**/*.json
release
.DS_Store
*.syso
coverage.txt

4
.jshintrc Normal file
View File

@@ -0,0 +1,4 @@
{
"asi": true,
"esnext": true
}

4
.mention-bot Normal file
View File

@@ -0,0 +1,4 @@
{
"numFilesToCheck": 10,
"requiredOrgs": ["minio"]
}

View File

@@ -1,24 +1,42 @@
sudo: false
go_import_path: github.com/minio/minio
sudo: required
services:
- docker
dist: trusty
language: go
os:
- linux
- osx
osx_image: xcode7.2
env:
- ARCH=x86_64
- ARCH=i686
script:
- make test
- make test GOFLAGS="-race"
## Run all the tests
- make
- diff -au <(gofmt -d cmd) <(printf "")
- diff -au <(gofmt -d pkg) <(printf "")
- make test GOFLAGS="-timeout 15m -race -v"
- make coverage
# Refer https://blog.hypriot.com/post/setup-simple-ci-pipeline-for-arm-images/
# push image
- >
if [ "$TRAVIS_BRANCH" == "master" ] && [ "$TRAVIS_PULL_REQUEST" == "false" ] && [ "$ARCH" == "x86_64" ]; then
docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build -t minio/minio:edge-armhf . -f Dockerfile.armhf
docker build -t minio/minio:edge-aarch64 . -f Dockerfile.aarch64
docker login -u="$DOCKER_USER" -p="$DOCKER_PASS"
docker push minio/minio:edge-armhf
docker push minio/minio:edge-aarch64
fi
after_success:
- bash <(curl -s https://codecov.io/bash)
go:
- 1.5.2
- 1.5.3
notifications:
slack:
secure: K9tsn5MvrCAxuEZTxn+m3Kq1K2NG2xMEJFSv/sTp+RQBW7TslPHzv859GsIvrm8mU1y1btOU9RlOzqrRUczI5cJpE8IL1oljPZbXrIXgetE0kbsw0Wpy99g27UQ2VGp933WDu8tfj7zU4cZv+BI0RltNLwqYO6GWXmcWP0IueCU=
- 1.7.5

View File

@@ -1,76 +0,0 @@
## Using aws-sdk-go with Minio
aws-sdk-go is the official AWS SDK for the Go programming language. This document covers
how to use aws-sdk-go with Minio server.
### Install AWS SDK S3 service
```sh
$ go get github.com/aws/aws-sdk-go/service/s3
```
### List all buckets on Minio
```go
package main
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
newSession := session.New()
s3Config := &aws.Config{
Credentials: credentials.NewStaticCredentials("<YOUR-ACCESS-KEY-ID>", "<YOUR-SECRET-ACCESS-KEY", ""),
Endpoint: aws.String("http://localhost:9000"),
Region: aws.String("us-east-1"),
DisableSSL: aws.Bool(true),
S3ForcePathStyle: aws.Bool(true),
}
// Create an S3 service object in the default region.
s3Client := s3.New(newSession, s3Config)
cparams := &s3.CreateBucketInput{
Bucket: aws.String("newbucket"), // Required
}
_, err := s3Client.CreateBucket(cparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
var lparams *s3.ListBucketsInput
// Call the ListBuckets() Operation
resp, err := s3Client.ListBuckets(lparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
// Pretty-print the response data.
fmt.Println(resp)
}
```
Populate your AccessKeyId and SecretAccessKey credentials and run the program as shown below.
```sh
$ go run aws-sdk-minio.go
{
Buckets: [{
CreationDate: 2015-10-22 01:46:04 +0000 UTC,
Name: "newbucket"
}],
Owner: {
DisplayName: "minio",
ID: "minio"
}
}
```

View File

@@ -1,39 +0,0 @@
## Minio Browser
Minio Browser uses Json Web Tokens to authenticate JSON RPC requests.
Initial request generates a token for 'AccessKey' and 'SecretKey'
provided by the user.
<blockquote>
Currently these tokens expire after 10hrs, this is not configurable yet.
</blockquote>
### Start minio server
```
minio server <testdir>
```
### JSON RPC APIs.
JSON RPC namespace is `Web`.
#### Auth Operations
* Login - waits for 'username, password' and on success replies a new Json Web Token (JWT).
* ResetToken - resets token, requires password and token.
* Logout - currently a dummy operation.
#### Bucket/Object Operations.
* ListBuckets - lists buckets, requires a valid token.
* ListObjects - lists objects, requires a valid token.
* MakeBucket - make a new bucket, requires a valid token.
* GetObjectURL - generates a URL for download access, requires a valid token.
(generated URL is valid for 1hr)
* PutObjectURL - generates a URL for upload access, requies a valid token.
(generated URL is valid for 1hr)
#### Server Operations.
* DiskInfo - get backend disk statistics.

View File

@@ -1,13 +1,13 @@
### Install Golang
If you do not have a working Golang environment setup please follow [Golang Installation Guide](./INSTALLGO.md).
If you do not have a working Golang environment setup please follow [Golang Installation Guide](https://docs.minio.io/docs/how-to-install-golang).
### Setup your Minio Github Repository
Fork [Minio upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL and pass it to ``go get`` command. Go uses git to clone a copy into your project workspace folder.
Fork [Minio upstream](https://github.com/minio/minio/fork) source repository to your own personal repository. Copy the URL for minio from your personal github repo (you will need it for the `git clone` command below).
```sh
$ mkdir -p $GOPATH/src/github.com/minio
$ cd $GOPATH/src/github.com/minio
$ git clone https://github.com/$USER_ID/minio
$ git clone <paste saved URL for personal forked minio repo>
$ cd minio
```
@@ -51,7 +51,7 @@ Building Libraries
- Push to the branch (git push origin my-new-feature)
- Create new Pull Request
* If you have additional dependencies for ``Minio``, ``Minio`` manages its depedencies using [govendor](https://github.com/kardianos/govendor)
* If you have additional dependencies for ``Minio``, ``Minio`` manages its dependencies using [govendor](https://github.com/kardianos/govendor)
- Run `go get foo/bar`
- Edit your code to import foo/bar
- Run `make pkg-add PKG=foo/bar` from top-level directory

View File

@@ -1,32 +0,0 @@
## Running Minio in Docker.
### Installing Docker.
```bash
sudo apt-get install Docker.io
```
### Generating `minio configs` for the first time.
```bash
docker run -p 9000:9001 minio/minio:latest
```
### Persist `minio configs`.
```bash
docker commit <running_minio_container_id> minio/my-minio
docker stop <running_minio_container_id>
```
### Create a data volume container.
```bash
docker create -v /export --name minio-export minio/my-minio /bin/true
```
You can then use the `--volumes-from` flag to mount the `/export` volume in another container.
```bash
docker run -p 9000:9001 --volumes-from minio-export --name minio1 minio/my-minio
```

View File

@@ -1,9 +1,30 @@
# use "make dockerimage" to build
FROM scratch
ENV DOCKERIMAGE 1
ADD minio.dockerimage /minio
ADD export /export
FROM alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
ENV GOPATH /go
ENV PATH $PATH:$GOPATH/bin
ENV CGO_ENABLED 0
WORKDIR /go/src/github.com/minio/
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps git go musl-dev && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
go get -v -d github.com/minio/minio && \
cd /go/src/github.com/minio/minio && \
go install -v -ldflags "$(go run buildscripts/gen-ldflags.go)" && \
rm -rf /go/pkg /go/src /usr/local/go && apk del .build-deps
EXPOSE 9000
EXPOSE 9001
ENTRYPOINT ["/minio"]
CMD ["server", "/export"]
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

30
Dockerfile.aarch64 Normal file
View File

@@ -0,0 +1,30 @@
FROM resin/aarch64-alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
ENV GOPATH /go
ENV PATH $PATH:$GOPATH/bin
ENV CGO_ENABLED 0
WORKDIR /go/src/github.com/minio/
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps git go musl-dev && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
go get -v -d github.com/minio/minio && \
cd /go/src/github.com/minio/minio && \
go install -v -ldflags "$(go run buildscripts/gen-ldflags.go)" && \
rm -rf /go/pkg /go/src /usr/local/go && apk del .build-deps
EXPOSE 9000
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

30
Dockerfile.armhf Normal file
View File

@@ -0,0 +1,30 @@
FROM resin/armhf-alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
ENV GOPATH /go
ENV PATH $PATH:$GOPATH/bin
ENV CGO_ENABLED 0
WORKDIR /go/src/github.com/minio/
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps git go musl-dev && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
go get -v -d github.com/minio/minio && \
cd /go/src/github.com/minio/minio && \
go install -v -ldflags "$(go run buildscripts/gen-ldflags.go)" && \
rm -rf /go/pkg /go/src /usr/local/go && apk del .build-deps
EXPOSE 9000
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

23
Dockerfile.release Normal file
View File

@@ -0,0 +1,23 @@
FROM alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps curl && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
curl https://dl.minio.io/server/minio/release/linux-amd64/minio > /usr/bin/minio && \
chmod +x /usr/bin/minio && apk del .build-deps
EXPOSE 9000
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

View File

@@ -0,0 +1,22 @@
FROM resin/aarch64-alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps curl && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
curl https://dl.minio.io/server/minio/release/linux-arm64/minio > /usr/bin/minio && \
chmod +x /usr/bin/minio && apk del .build-deps
EXPOSE 9000
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

22
Dockerfile.release.armhf Normal file
View File

@@ -0,0 +1,22 @@
FROM resin/armhf-alpine:3.5
MAINTAINER Minio Inc <dev@minio.io>
RUN \
apk add --no-cache ca-certificates && \
apk add --no-cache --virtual .build-deps curl && \
echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
curl https://dl.minio.io/server/minio/release/linux-arm/minio > /usr/bin/minio && \
chmod +x /usr/bin/minio && apk del .build-deps
EXPOSE 9000
COPY buildscripts/docker-entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/docker-entrypoint.sh
ENTRYPOINT ["/usr/bin/docker-entrypoint.sh"]
VOLUME ["/export"]
CMD ["minio"]

View File

@@ -1,84 +0,0 @@
## Ubuntu (Kylin) 14.04
### Build Dependencies
This installation document assumes Ubuntu 14.04+ on x86-64 platform.
##### Install Git, GCC
```sh
$ sudo apt-get install git build-essential
```
##### Install Go 1.5+
Download Go 1.5+ from [https://golang.org/dl/](https://golang.org/dl/).
```sh
$ wget https://storage.googleapis.com/golang/go1.5.1.linux-amd64.tar.gz
$ mkdir -p ${HOME}/bin/
$ mkdir -p ${HOME}/go/
$ tar -C ${HOME}/bin/ -xzf go1.5.1.linux-amd64.tar.gz
```
##### Setup GOROOT and GOPATH
Add the following exports to your ``~/.bashrc``. Environment variable GOROOT specifies the location of your golang binaries
and GOPATH specifies the location of your project workspace.
```sh
export GOROOT=${HOME}/bin/go
export GOPATH=${HOME}/go
export PATH=$PATH:${HOME}/bin/go/bin:${GOPATH}/bin
```
##### Source the new enviornment
```sh
$ source ~/.bashrc
```
##### Testing it all
```sh
$ go env
```
## OS X (Yosemite) 10.10
### Build Dependencies
This installation document assumes OS X Yosemite 10.10+ on x86-64 platform.
##### Install brew
```sh
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```
##### Install Git, Python
```sh
$ brew install git python
```
##### Install Go 1.5+
Install golang binaries using `brew`
```sh
$ brew install go
$ mkdir -p $HOME/go
```
##### Setup GOROOT and GOPATH
Add the following exports to your ``~/.bash_profile``. Environment variable GOROOT specifies the location of your golang binaries
and GOPATH specifies the location of your project workspace.
```sh
export GOPATH=${HOME}/go
export GOVERSION=$(brew list go | head -n 1 | cut -d '/' -f 6)
export GOROOT=$(brew --prefix)/Cellar/go/${GOVERSION}/libexec
export PATH=$PATH:${GOPATH}/bin
```
##### Source the new enviornment
```sh
$ source ~/.bash_profile
```
##### Testing it all
```sh
$ go env
```

View File

@@ -18,7 +18,7 @@ $
- Run `go get foo/bar`
- Edit your code to import foo/bar
- Run `govendor add foo/bar` from top-level folder
- Run `govendor add foo/bar` from top-level directory
#### Remove dependencies
@@ -34,4 +34,4 @@ $
`minio` doesn't follow semantic versioning style, `minio` instead uses the release date and time as the release versions.
`make release` will install new released binary into your `GOPATH`
`make release` will generate new binary into `release` directory.

177
Makefile
View File

@@ -1,103 +1,148 @@
LDFLAGS := $(shell go run buildscripts/gen-ldflags.go)
DOCKER_BIN := $(shell which docker)
DOCKER_LDFLAGS := '$(LDFLAGS) -extldflags "-static"'
PWD := $(shell pwd)
GOPATH := $(shell go env GOPATH)
BUILD_LDFLAGS := '$(LDFLAGS)'
TAG := latest
UI_ASSETS := ui-assets.go
UI_ASSETS_ARMOR := ui-assets.asc
HOST ?= $(shell uname)
CPU ?= $(shell uname -m)
# if no host is identifed (no uname tool)
# we assume a Linux-64bit build
ifeq ($(HOST),)
HOST = Linux
endif
# identify CPU
ifeq ($(CPU), x86_64)
HOST := $(HOST)64
else
ifeq ($(CPU), amd64)
HOST := $(HOST)64
else
ifeq ($(CPU), i686)
HOST := $(HOST)32
endif
endif
endif
#############################################
# now we find out the target OS for
# which we are going to compile in case
# the caller didn't yet define OS himself
ifndef (OS)
ifeq ($(HOST), Linux64)
arch = gcc
else
ifeq ($(HOST), Linux32)
arch = 32
else
ifeq ($(HOST), Darwin64)
arch = clang
else
ifeq ($(HOST), Darwin32)
arch = clang
else
ifeq ($(HOST), FreeBSD64)
arch = gcc
endif
endif
endif
endif
endif
endif
all: install
checkdeps:
@echo "Checking deps:"
checks:
@echo "Check deps"
@(env bash $(PWD)/buildscripts/checkdeps.sh)
@echo "Checking project is in GOPATH"
@(env bash $(PWD)/buildscripts/checkgopath.sh)
checkgopath:
@echo "Checking if project is at ${GOPATH}"
@for miniopath in $(echo ${GOPATH} | sed 's/:/\n/g'); do if [ ! -d ${miniopath}/src/github.com/minio/minio ]; then echo "Project not found in ${miniopath}, please follow instructions provided at https://github.com/minio/minio/blob/master/CONTRIBUTING.md#setup-your-minio-github-repository" && exit 1; fi done
@echo "Setting BUILD_LDFLAGS: ${BUILD_LDFLAGS}"
getdeps: checks
@echo "Installing golint" && go get -u github.com/golang/lint/golint
@echo "Installing gocyclo" && go get -u github.com/fzipp/gocyclo
@echo "Installing deadcode" && go get -u github.com/remyoudompheng/go-misc/deadcode
@echo "Installing misspell" && go get -u github.com/client9/misspell/cmd/misspell
@echo "Installing ineffassign" && go get -u github.com/gordonklaus/ineffassign
checkdocker:
@echo "Checking if docker is installed.. "
@if [ -z ${DOCKER_BIN} ]; then echo "Docker not installed, cannot build docker image. Please install 'sudo apt-get install docker.io'" && exit 1; else echo "Docker installed at ${DOCKER_BIN}."; fi;
getdeps: checkdeps checkgopath
@go get -u github.com/golang/lint/golint && echo "Installed golint:"
@go get -u golang.org/x/tools/cmd/vet && echo "Installed vet:"
@go get -u github.com/fzipp/gocyclo && echo "Installed gocyclo:"
@go get -u github.com/remyoudompheng/go-misc/deadcode && echo "Installed deadcode:"
$(UI_ASSETS):
@curl -s https://dl.minio.io/assets/server/$(UI_ASSETS_ARMOR) 2>&1 > $(UI_ASSETS_ARMOR) && echo "Downloading signature file $(UI_ASSETS_ARMOR) for verification:"
@gpg --batch --no-tty --yes --keyserver pgp.mit.edu --recv-keys F9AAC728 2>&1 > /dev/null && echo "Importing public key:"
@curl -s https://dl.minio.io/assets/server/$@ 2>&1 > $@ && echo "Downloading UI assets file $@:"
@gpg --batch --no-tty --verify $(UI_ASSETS_ARMOR) $@ 2>&1 > /dev/null && echo "Verifying signature of downloaded assets."
verifiers: getdeps vet fmt lint cyclo
verifiers: vet fmt lint cyclo spelling
vet:
@echo "Running $@:"
@GO15VENDOREXPERIMENT=1 go tool vet -all *.go
@GO15VENDOREXPERIMENT=1 go tool vet -all ./pkg
@GO15VENDOREXPERIMENT=1 go tool vet -shadow=true *.go
@GO15VENDOREXPERIMENT=1 go tool vet -shadow=true ./pkg
@echo "Running $@"
@go tool vet -atomic -bool -copylocks -nilfunc -printf -shadow -rangeloops -unreachable -unsafeptr -unusedresult cmd
@go tool vet -atomic -bool -copylocks -nilfunc -printf -shadow -rangeloops -unreachable -unsafeptr -unusedresult pkg
fmt:
@echo "Running $@:"
@GO15VENDOREXPERIMENT=1 gofmt -s -l *.go
@GO15VENDOREXPERIMENT=1 gofmt -s -l pkg
@echo "Running $@"
@gofmt -d cmd
@gofmt -d pkg
lint:
@echo "Running $@:"
@GO15VENDOREXPERIMENT=1 ${GOPATH}/bin/golint *.go
@GO15VENDOREXPERIMENT=1 ${GOPATH}/bin/golint github.com/minio/minio/pkg...
@echo "Running $@"
@${GOPATH}/bin/golint -set_exit_status github.com/minio/minio/cmd...
@${GOPATH}/bin/golint -set_exit_status github.com/minio/minio/pkg...
ineffassign:
@echo "Running $@"
@${GOPATH}/bin/ineffassign .
cyclo:
@echo "Running $@:"
@GO15VENDOREXPERIMENT=1 ${GOPATH}/bin/gocyclo -over 65 *.go
@GO15VENDOREXPERIMENT=1 ${GOPATH}/bin/gocyclo -over 65 pkg
@echo "Running $@"
@${GOPATH}/bin/gocyclo -over 100 cmd
@${GOPATH}/bin/gocyclo -over 100 pkg
build: getdeps verifiers $(UI_ASSETS)
@echo "Installing minio:"
deadcode:
@GO15VENDOREXPERIMENT=1 deadcode
@${GOPATH}/bin/deadcode
spelling:
@${GOPATH}/bin/misspell -error `find cmd/`
@${GOPATH}/bin/misspell -error `find pkg/`
@${GOPATH}/bin/misspell -error `find docs/`
test: build
@echo "Running all testing:"
@GO15VENDOREXPERIMENT=1 go test $(GOFLAGS) .
@GO15VENDOREXPERIMENT=1 go test $(GOFLAGS) github.com/minio/minio/pkg...
@echo "Running all minio testing"
@go test $(GOFLAGS) .
@go test $(GOFLAGS) github.com/minio/minio/cmd...
@go test $(GOFLAGS) github.com/minio/minio/pkg...
coverage: build
@echo "Running all coverage for minio"
@./buildscripts/go-coverage.sh
gomake-all: build
@GO15VENDOREXPERIMENT=1 go build --ldflags $(BUILD_LDFLAGS) -o $(GOPATH)/bin/minio
@echo "Installing minio at $(GOPATH)/bin/minio"
@go build --ldflags $(BUILD_LDFLAGS) -o $(GOPATH)/bin/minio
pkg-add:
@GO15VENDOREXPERIMENT=1 govendor add $(PKG)
@echo "Adding new package $(PKG)"
@${GOPATH}/bin/govendor add $(PKG)
pkg-update:
@GO15VENDOREXPERIMENT=1 govendor update $(PKG)
@echo "Updating new package $(PKG)"
@${GOPATH}/bin/govendor update $(PKG)
pkg-remove:
@GO15VENDOREXPERIMENT=1 govendor remove $(PKG)
@echo "Remove new package $(PKG)"
@${GOPATH}/bin/govendor remove $(PKG)
pkg-list:
@$(GOPATH)/bin/govendor list
install: gomake-all
dockerimage: checkdocker verifiers $(UI_ASSETS)
@echo "Building docker image:" minio:$(TAG)
@GO15VENDOREXPERIMENT=1 go build --ldflags $(DOCKER_LDFLAGS) -o minio.dockerimage
@mkdir -p export
@sudo docker build --rm --tag=minio/minio:$(TAG) .
@rmdir export
@rm minio.dockerimage
release: verifiers
@MINIO_RELEASE=RELEASE ./buildscripts/build.sh
release:
@./release.sh
experimental: verifiers
@MINIO_RELEASE=EXPERIMENTAL ./buildscripts/build.sh
clean:
@echo "Cleaning up all the generated files:"
@rm -fv cover.out
@rm -fv minio
@rm -fv minio.test
@rm -fv pkg/fs/fs.test
@rm -fv ui-assets.go
@rm -fv ui-assets.asc
@echo "Cleaning up all the generated files"
@find . -name '*.test' | xargs rm -fv
@rm -rf build
@rm -rf release

2
NOTICE
View File

@@ -5,5 +5,5 @@ This product includes software developed at Minio, Inc.
The Minio project contains unmodified/modified subcomponents too with
separate copyright notices and license terms. Your use of the source
code for the these subcomponents is subject to the terms and conditions
code for these subcomponents is subject to the terms and conditions
of the following licenses.

266
README.md
View File

@@ -1,200 +1,122 @@
## Minio [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/minio/minio?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
# Minio Quickstart Guide [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Go Report Card](https://goreportcard.com/badge/minio/minio)](https://goreportcard.com/report/minio/minio) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800)](https://hub.docker.com/r/minio/minio/) [![codecov](https://codecov.io/gh/minio/minio/branch/master/graph/badge.svg)](https://codecov.io/gh/minio/minio)
Minio is a distributed object storage server written in Golang. Source is available under free software / open source [Apache license 2.0](./LICENSE). API compatible with Amazon S3 cloud storage service.
Minio is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.
## Description
Micro services environment provisions one Minio server per application instance. Scalability is achieved through large number of smaller personalized instances. This version of the Minio binary is built using Filesystem storage backend for magnetic and solid state disks.
## Minio Client
[Minio Client (mc)](https://github.com/minio/mc#minio-client-mc-) provides a modern alternative to Unix commands like ``ls``, ``cat``, ``cp``, ``sync``, and ``diff``. It supports POSIX compatible filesystems and Amazon S3 compatible cloud storage systems. It is entirely written in Golang.
## Amazon S3 Compatible Client Libraries
- [Golang Library](https://github.com/minio/minio-go)
- [Java Library](https://github.com/minio/minio-java)
- [Nodejs Library](https://github.com/minio/minio-js)
- [Python Library](https://github.com/minio/minio-py)
- [.Net Library](https://github.com/minio/minio-dotnet)
### Install [![Build Status](https://travis-ci.org/minio/minio.svg?branch=master)](https://travis-ci.org/minio/minio)[![Build status](https://ci.appveyor.com/api/projects/status/royh137dni8yevep/branch/master?svg=true)](https://ci.appveyor.com/project/harshavardhana/minio-qxbjq/branch/master)
#### GNU/Linux
Download ``minio`` for:
- ``64-bit Intel`` from https://dl.minio.io/server/minio/release/linux-amd64/minio
- ``32-bit Intel`` from https://dl.minio.io/server/minio/release/linux-386/minio
- ``32-bit ARM`` from https://dl.minio.io/server/minio/release/linux-arm/minio
~~~
$ chmod +x minio
$ ./minio --help
~~~
#### OS X
Download ``minio`` from https://dl.minio.io/server/minio/release/darwin-amd64/minio
~~~
$ chmod 755 minio
$ ./minio --help
~~~
#### Microsoft Windows
Download ``minio`` for:
- ``64-bit`` from https://dl.minio.io/server/minio/release/windows-amd64/minio.exe
- ``32-bit`` from https://dl.minio.io/server/minio/release/windows-386/minio.exe
~~~
C:\Users\Username\Downloads> minio.exe --help
~~~
#### Docker container
Download ``minio`` for docker.
~~~
$ docker pull minio/minio
~~~
Read more here on [How to configure data volume containers for Minio?](./Docker.md)
#### Source
<blockquote>
NOTE: Source installation is intended for only developers and advanced users. For general use, please download official releases from https://minio.io/download.
</blockquote>
If you do not have a working Golang environment, please follow [Install Golang](./INSTALLGO.md).
~~~
$ go get -d github.com/minio/minio
$ cd $GOPATH/src/github.com/minio/minio
$ make
~~~
### How to use Minio?
Minio server is light enough to be bundled with the application stack, similar to NodeJS, Redis and MySQL.
## Docker Container
### Stable
```
NAME:
minio server - Start Minio cloud storage server.
USAGE:
minio server [OPTION VALUE] PATH
OPTION = expiry VALUE = NN[h|m|s] [DEFAULT=Unlimited]
OPTION = min-free-disk VALUE = NN% [DEFAULT: 10%]
EXAMPLES:
1. Start minio server on Linux.
$ minio server /home/shared
2. Start minio server on Windows.
$ minio server C:\MyShare
3. Start minio server bound to a specific IP:PORT, when you have multiple network interfaces.
$ minio --address 192.168.1.101:9000 server /home/shared
4. Start minio server with minimum free disk threshold to 5%
$ minio server min-free-disk 5% /home/shared/Pictures
5. Start minio server with minimum free disk threshold to 15% with auto expiration set to 1h
$ minio server min-free-disk 15% expiry 1h /home/shared/Documents
docker pull minio/minio
docker run -p 9000:9000 minio/minio server /export
```
#### Start Minio server.
### Edge
```
docker pull minio/minio:edge
docker run -p 9000:9000 minio/minio:edge server /export
```
Please visit Minio Docker quickstart guide for more [here](https://docs.minio.io/docs/minio-docker-quickstart-guide)
~~~
$ minio server ~/Photos
AccessKey: WLGDGYAQYIGI833EV05A SecretKey: BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF Region: us-east-1
## macOS
### Homebrew
Install minio packages using [Homebrew](http://brew.sh/)
Minio Object Storage:
http://127.0.0.1:9000
http://10.0.0.3:9000
Minio Browser:
http://127.0.0.1:9000
http://10.0.0.3:9000
To configure Minio Client:
$ wget https://dl.minio.io/client/mc/release/darwin-amd64/mc
$ chmod 755 mc
$ ./mc config host add myminio http://localhost:9000 WLGDGYAQYIGI833EV05A BYvgJM101sHngl2uzjXS/OBF/aMxAN06JrJ3qJlF
~~~
#### How to use AWS CLI with Minio?
<blockquote>
This section assumes that you have already installed aws-cli, if not please visit https://aws.amazon.com/cli/
</blockquote>
To configure `aws-cli`, type `aws configure` and follow below steps.
```sh
brew install minio/stable/minio
minio server ~/Photos
```
#### Note
If you previously installed minio using `brew install minio` then uninstall minio as shown below
```
$ aws configure
AWS Access Key ID [None]: YOUR_ACCESS_KEY_HERE
AWS Secret Access Key [None]: YOUR_SECRET_KEY_HERE
Default region name [None]: us-east-1
Default output format [None]: ENTER
brew uninstall minio
```
Additionally enable `aws-cli` to use AWS Signature Version '4' for Minio server.
Then re-install the latest minio using:
```
$ aws configure set default.s3.signature_version s3v4
brew install minio/stable/minio
```
>`brew install minio` and `brew upgrade minio` will no longer install/upgrade the latest minio binaries on macOS. Upstream bugs in golang 1.8 broke Minio brew installer. Use the updated `minio/stable/minio` in your brew paths.
### Binary Download
| Platform| Architecture | URL|
| ----------| -------- | ------|
|Apple macOS|64-bit Intel|https://dl.minio.io/server/minio/release/darwin-amd64/minio |
```sh
chmod 755 minio
./minio server ~/Photos
```
To list your buckets.
```
$ aws --endpoint-url http://localhost:9000 s3 ls
2016-01-07 16:38:23 testbucket
## GNU/Linux
### Binary Download
| Platform| Architecture | URL|
| ----------| -------- | ------|
|GNU/Linux|64-bit Intel|https://dl.minio.io/server/minio/release/linux-amd64/minio |
| |32-bit Intel|https://dl.minio.io/server/minio/release/linux-386/minio |
| |32-bit ARM|https://dl.minio.io/server/minio/release/linux-arm/minio |
| |64-bit ARM|https://dl.minio.io/server/minio/release/linux-arm64/minio |
| |32-bit ARMv6|https://dl.minio.io/server/minio/release/linux-arm6vl/minio |
```sh
chmod +x minio
./minio server ~/Photos
```
To list contents inside bucket.
```
$ aws --endpoint-url http://localhost:9000 s3 ls s3://testbucket
PRE test/
2015-12-17 08:46:41 12232928 vim
2016-01-07 16:38:23 32232928 emacs
2015-12-09 08:05:24 138504 s3cmd
## Microsoft Windows
### Binary Download
| Platform| Architecture | URL|
| ----------| -------- | ------|
|Microsoft Windows|64-bit|https://dl.minio.io/server/minio/release/windows-amd64/minio.exe |
| |32-bit|https://dl.minio.io/server/minio/release/windows-386/minio.exe |
```sh
minio.exe server D:\Photos
```
#### How to use AWS SDK with Minio?
## FreeBSD
### Port
Install minio packages using [pkg](https://github.com/freebsd/pkg)
Please follow the documentation here - [Using aws-sdk-go with Minio server](./AWS-SDK-GO.md)
#### How to use s3cmd with Minio?
<blockquote>
This section assumes that you have already installed s3cmd, if not please visit http://s3tools.org/s3cmd
</blockquote>
Edit the following fields in your s3cmd configuration file `~/.s3cfg` .
```
host_base = localhost:9000
host_bucket = localhost:9000
access_key = YOUR_ACCESS_KEY_HERE
secret_key = YOUR_SECRET_KEY_HERE
```sh
pkg install minio
sysrc minio_enable=yes
sysrc minio_disks=/home/user/Photos
service minio start
```
To list your buckets.
```
$ s3cmd ls s3://
2015-12-09 16:12 s3://testbbucket
### Binary Download
| Platform| Architecture | URL|
| ----------| -------- | ------|
|FreeBSD|64-bit|https://dl.minio.io/server/minio/release/freebsd-amd64/minio |
```sh
chmod 755 minio
./minio server ~/Photos
```
To list contents inside buckets.
```
$ s3cmd ls s3://testbucket/
DIR s3://testbucket/test/
2015-12-09 16:05 138504 s3://testbucket/newfile
## Install from Source
Source installation is only intended for developers and advanced users. If you do not have a working Golang environment, please follow [How to install Golang](https://docs.minio.io/docs/how-to-install-golang).
```sh
go get -u github.com/minio/minio
```
## Test using Minio Browser
Minio Server comes with an embedded web based object browser. Point your web browser to http://127.0.0.1:9000 ensure your server has started successfully.
![Screenshot](https://github.com/minio/minio/blob/master/docs/screenshots/minio-browser.jpg?raw=true)
## Test using Minio Client `mc`
`mc` provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage services. Follow the Minio Client [Quickstart Guide](https://docs.minio.io/docs/minio-client-quickstart-guide) for further instructions.
## Explore Further
- [Minio Erasure Code QuickStart Guide](https://docs.minio.io/docs/minio-erasure-code-quickstart-guide)
- [Use `mc` with Minio Server](https://docs.minio.io/docs/minio-client-quickstart-guide)
- [Use `aws-cli` with Minio Server](https://docs.minio.io/docs/aws-cli-with-minio)
- [Use `s3cmd` with Minio Server](https://docs.minio.io/docs/s3cmd-with-minio)
- [Use `minio-go` SDK with Minio Server](https://docs.minio.io/docs/golang-client-quickstart-guide)
- [The Minio documentation website](https://docs.minio.io)
## Contribute to Minio Project
Please follow Minio [Contributor's Guide](./CONTRIBUTING.md)
### Jobs
If you think in Lisp or Haskell and hack in go, you would blend right in. Send your github link to callhome@minio.io.
Please follow Minio [Contributor's Guide](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)

202
README_ZH.md Normal file
View File

@@ -0,0 +1,202 @@
# Minio 快速入门 [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Go Report Card](https://goreportcard.com/badge/minio/minio)](https://goreportcard.com/report/minio/minio) [![codecov](https://codecov.io/gh/minio/minio/branch/master/graph/badge.svg)](https://codecov.io/gh/minio/minio)
Minio是一个对象存储服务基于Apache License v2.0协议. 它完全兼容亚马逊的S3云储存服务非常适合于存储很多非结构化的数据例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等而一个对象文件可以是任意大小从几kb到最大5T不等。
## 1. 下载
Minio是一个非常轻量的服务,可以很简单的和其他应用的结合,类似 NodeJS, Redis 或者 MySQL.
| Platform| Architecture | URL|
| ----------| -------- | ------|
|GNU/Linux|64-bit Intel|https://dl.minio.io/server/minio/release/linux-amd64/minio|
||32-bit Intel|https://dl.minio.io/server/minio/release/linux-386/minio|
||32-bit ARM|https://dl.minio.io/server/minio/release/linux-arm/minio|
||64-bit ARM|https://dl.minio.io/server/minio/release/linux-arm64/minio|
||32-bit ARMv6|https://dl.minio.io/server/minio/release/linux-arm6vl/minio|
|Apple OS X|64-bit Intel|https://dl.minio.io/server/minio/release/darwin-amd64/minio|
|Microsoft Windows|64-bit|https://dl.minio.io/server/minio/release/windows-amd64/minio.exe|
||32-bit|https://dl.minio.io/server/minio/release/windows-386/minio.exe|
|FreeBSD|64-bit|https://dl.minio.io/server/minio/release/freebsd-amd64/minio|
### Homebrew 安装
使用[Homebrew](http://brew.sh/) 来安装minio
```sh
$ brew install minio
$ minio --help
```
### 源码安装
源码安装只针对开发者和一些高级用户如果你还没有golang的环境请安装golang官网安装[How to install Golang](https://docs.minio.io/docs/zh-CN/how-to-install-golang).
```sh
$ go get -u github.com/minio/minio
```
## 2. 运行Minio服务
### GNU/Linux
```sh
$ chmod +x minio
$ ./minio --help
$ ./minio server ~/Photos
端点: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
区域: us-east-1
浏览器访问入口:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
命令行访问: https://docs.minio.io/docs/zh-CN/minio-client-quickstart-guide
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
对象操作API (兼容Amazon S3):
Go: https://docs.minio.io/docs/zh-CN/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/zh-CN/java-client-quickstart-guide
Python: https://docs.minio.io/docs/zh-CN/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/zh-CN/javascript-client-quickstart-guide
```
### OS X
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
端点: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
区域: us-east-1
浏览器访问入口:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
命令行访问: https://docs.minio.io/docs/zh-CN/minio-client-quickstart-guide
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
对象操作API (兼容Amazon S3):
Go: https://docs.minio.io/docs/zh-CN/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/zh-CN/java-client-quickstart-guide
Python: https://docs.minio.io/docs/zh-CN/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/zh-CN/javascript-client-quickstart-guide
```
### Microsoft Windows
```sh
C:\Users\Username\Downloads> minio.exe --help
C:\Users\Username\Downloads> minio.exe server D:\Photos
端点: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
区域: us-east-1
浏览器访问入口:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
命令行访问: https://docs.minio.io/docs/zh-CN/minio-client-quickstart-guide
$ mc.exe config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
对象操作API (兼容Amazon S3):
Go: https://docs.minio.io/docs/zh-CN/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/zh-CN/java-client-quickstart-guide
Python: https://docs.minio.io/docs/zh-CN/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/zh-CN/javascript-client-quickstart-guide
```
### Docker
```sh
$ docker pull minio/minio
$ docker run -p 9000:9000 minio/minio
```
访问minio的docker入门指南获得更多内容 [here](https://docs.minio.io/docs/zh-CN/minio-docker-quickstart-guide)
### FreeBSD
```sh
$ chmod 755 minio
$ ./minio --help
$ ./minio server ~/Photos
端点: http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
AccessKey: USWUXHGYZQYFYFFIT3RE
SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
区域: us-east-1
浏览器访问入口:
http://10.0.0.10:9000 http://127.0.0.1:9000 http://172.17.0.1:9000
命令行访问: https://docs.minio.io/docs/zh-CN/minio-client-quickstart-guide
$ mc config host add myminio http://10.0.0.10:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03
对象操作API (兼容Amazon S3):
Go: https://docs.minio.io/docs/zh-CN/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/zh-CN/java-client-quickstart-guide
Python: https://docs.minio.io/docs/zh-CN/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/zh-CN/javascript-client-quickstart-guide
```
请访问FreeBSD的官网指南获取更多详细信息[here](https://www.freebsd.org/doc/handbook/zfs-quickstart.html)
## 3. 使用浏览器测试minio服务
打开浏览器并输入 http://127.0.0.1:9000 查看在minio服务器上面的所有bucket
![Screenshot](https://github.com/minio/minio/blob/master/docs/screenshots/minio-browser.jpg?raw=true)
## 4. 使用`mc`测试minio服务
按照 [这个](https://docs.minio.io/docs/minio-client-quickstart-guide) 安装mc. 使用 `mc ls` 命令显示所有在minio服务上面的bucket.
```sh
$ mc ls myminio/
[2015-08-05 08:13:22 IST] 0B andoria/
[2015-08-05 06:14:26 IST] 0B deflector/
[2015-08-05 08:13:11 IST] 0B ferenginar/
[2016-03-08 14:56:35 IST] 0B jarjarbing/
[2016-01-20 16:07:41 IST] 0B my.minio.io/
```
查看更多的例子请访问 [Minio Client Complete Guide](https://docs.minio.io/docs/zh-CN/minio-client-complete-guide).
## 5. 更多内容
- [Minio Erasure Code 快速入门](https://docs.minio.io/docs/zh-CN/minio-erasure-code-quickstart-guide)
- [Minio Docker 快速入门](https://docs.minio.io/docs/zh-CN/minio-docker-quickstart-guide)
- [使用`mc`测试 Minio Server](https://docs.minio.io/docs/zh-CN/minio-client-quickstart-guide)
- [使用 `aws-cli` 测试 Minio Server](https://docs.minio.io/docs/zh-CN/aws-cli-with-minio)
- [使用 `s3cmd` 测试 Minio Server](https://docs.minio.io/docs/zh-CN/s3cmd-with-minio)
- [使用 `minio-go` SDK ce's测试 Minio Server](https://docs.minio.io/docs/zh-CN/golang-client-quickstart-guide)
## 6. 给Minio项目贡献
请按照Minio [贡献者指导手册](https://github.com/minio/minio/blob/master/CONTRIBUTING.md)

View File

@@ -1,120 +0,0 @@
/*
* Minimalist Object Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"encoding/json"
"net/http"
"net/url"
"os"
"time"
"github.com/minio/minio-xl/pkg/probe"
)
type accessLogHandler struct {
http.Handler
accessLogFile *os.File
}
// LogMessage is a serializable json log message
type LogMessage struct {
StartTime time.Time
Duration time.Duration
StatusMessage string // human readable http status message
ContentLength string // human readable content length
// HTTP detailed message
HTTP struct {
ResponseHeaders http.Header
Request struct {
Method string
URL *url.URL
Proto string // "HTTP/1.0"
ProtoMajor int // 1
ProtoMinor int // 0
Header http.Header
Host string
Form url.Values
PostForm url.Values
Trailer http.Header
RemoteAddr string
RequestURI string
}
}
}
func (h *accessLogHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
message, err := getLogMessage(w, req)
fatalIf(err.Trace(), "Unable to extract http message.", nil)
_, e := h.accessLogFile.Write(message)
fatalIf(probe.NewError(e), "Writing to log file failed.", nil)
h.Handler.ServeHTTP(w, req)
}
func getLogMessage(w http.ResponseWriter, req *http.Request) ([]byte, *probe.Error) {
logMessage := &LogMessage{
StartTime: time.Now().UTC(),
}
// store lower level details
logMessage.HTTP.ResponseHeaders = w.Header()
logMessage.HTTP.Request = struct {
Method string
URL *url.URL
Proto string // "HTTP/1.0"
ProtoMajor int // 1
ProtoMinor int // 0
Header http.Header
Host string
Form url.Values
PostForm url.Values
Trailer http.Header
RemoteAddr string
RequestURI string
}{
Method: req.Method,
URL: req.URL,
Proto: req.Proto,
ProtoMajor: req.ProtoMajor,
ProtoMinor: req.ProtoMinor,
Header: req.Header,
Host: req.Host,
Form: req.Form,
PostForm: req.PostForm,
Trailer: req.Header,
RemoteAddr: req.RemoteAddr,
RequestURI: req.RequestURI,
}
// logMessage.HTTP.Request = req
logMessage.Duration = time.Now().UTC().Sub(logMessage.StartTime)
js, e := json.Marshal(logMessage)
if e != nil {
return nil, probe.NewError(e)
}
js = append(js, byte('\n')) // append a new line
return js, nil
}
// setAccessLogHandler logs requests
func setAccessLogHandler(h http.Handler) http.Handler {
file, e := os.OpenFile("access.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600)
fatalIf(probe.NewError(e), "Unable to open access log.", nil)
return &accessLogHandler{Handler: h, accessLogFile: file}
}

View File

@@ -1,76 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"crypto/rand"
"encoding/base64"
"regexp"
"github.com/minio/minio-xl/pkg/probe"
)
const (
minioAccessID = 20
minioSecretID = 40
)
// isValidAccessKey - validate access key
func isValidAccessKey(accessKeyID string) bool {
if accessKeyID == "" {
return true
}
regex := regexp.MustCompile("^[A-Z0-9\\-\\.\\_\\~]{20}$")
return regex.MatchString(accessKeyID)
}
// generateAccessKeyID - generate random alpha numeric value using only uppercase characters
// takes input as size in integer
func generateAccessKeyID() ([]byte, *probe.Error) {
alpha := make([]byte, minioAccessID)
if _, e := rand.Read(alpha); e != nil {
return nil, probe.NewError(e)
}
for i := 0; i < minioAccessID; i++ {
alpha[i] = alphaNumericTable[alpha[i]%byte(len(alphaNumericTable))]
}
return alpha, nil
}
// generateSecretAccessKey - generate random base64 numeric value from a random seed.
func generateSecretAccessKey() ([]byte, *probe.Error) {
rb := make([]byte, minioSecretID)
if _, e := rand.Read(rb); e != nil {
return nil, probe.NewError(e)
}
return []byte(base64.StdEncoding.EncodeToString(rb))[:minioSecretID], nil
}
// mustGenerateAccessKeyID - must generate random alpha numeric value using only uppercase characters
// takes input as size in integer
func mustGenerateAccessKeyID() []byte {
alpha, err := generateAccessKeyID()
fatalIf(err.Trace(), "Unable to generate accessKeyID.", nil)
return alpha
}
// mustGenerateSecretAccessKey - generate random base64 numeric value from a random seed.
func mustGenerateSecretAccessKey() []byte {
secretKey, err := generateSecretAccessKey()
fatalIf(err.Trace(), "Unable to generate secretAccessKey.", nil)
return secretKey
}

View File

@@ -1,225 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import "encoding/xml"
// Limit number of objects in a given response.
const (
maxObjectList = 1000
)
// LocationResponse - format for location response.
type LocationResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ LocationConstraint" json:"-"`
Location string `xml:",chardata"`
}
// AccessControlPolicyResponse - format for get bucket acl response.
type AccessControlPolicyResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ AccessControlPolicy" json:"-"`
AccessControlList struct {
Grant []Grant
}
Owner Owner
}
// Grant container for grantee and permission.
type Grant struct {
Grantee struct {
ID string
DisplayName string
EmailAddress string
Type string
URI string
}
Permission string
}
// ListObjectsResponse - format for list objects response.
type ListObjectsResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ ListBucketResult" json:"-"`
CommonPrefixes []*CommonPrefix
Contents []*Object
Delimiter string
// Encoding type used to encode object keys in the response.
EncodingType string
// A flag that indicates whether or not ListObjects returned all of the results
// that satisfied the search criteria.
IsTruncated bool
Marker string
MaxKeys int
Name string
// When response is truncated (the IsTruncated element value in the response
// is true), you can use the key name in this field as marker in the subsequent
// request to get next set of objects. Server lists objects in alphabetical
// order Note: This element is returned only if you have delimiter request parameter
// specified. If response does not include the NextMaker and it is truncated,
// you can use the value of the last Key in the response as the marker in the
// subsequent request to get the next set of object keys.
NextMarker string
Prefix string
}
// Part container for part metadata.
type Part struct {
PartNumber int
ETag string
LastModified string
Size int64
}
// ListPartsResponse - format for list parts response.
type ListPartsResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ ListPartsResult" json:"-"`
Bucket string
Key string
UploadID string `xml:"UploadId"`
Initiator Initiator
Owner Owner
// The class of storage used to store the object.
StorageClass string
PartNumberMarker int
NextPartNumberMarker int
MaxParts int
IsTruncated bool
// List of parts.
Part []*Part
}
// ListMultipartUploadsResponse - format for list multipart uploads response.
type ListMultipartUploadsResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ ListMultipartUploadsResult" json:"-"`
Bucket string
KeyMarker string
UploadIDMarker string `xml:"UploadIdMarker"`
NextKeyMarker string
NextUploadIDMarker string `xml:"NextUploadIdMarker"`
EncodingType string
MaxUploads int
IsTruncated bool
Upload []*Upload
Prefix string
Delimiter string
CommonPrefixes []*CommonPrefix
}
// ListBucketsResponse - format for list buckets response
type ListBucketsResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ ListAllMyBucketsResult" json:"-"`
// Container for one or more buckets.
Buckets struct {
Bucket []*Bucket
} // Buckets are nested
Owner Owner
}
// Upload container for in progress multipart upload
type Upload struct {
Key string
UploadID string `xml:"UploadId"`
Initiator Initiator
Owner Owner
StorageClass string
Initiated string
}
// CommonPrefix container for prefix response in ListObjectsResponse
type CommonPrefix struct {
Prefix string
}
// Bucket container for bucket metadata
type Bucket struct {
Name string
CreationDate string
}
// Object container for object metadata
type Object struct {
ETag string
Key string
LastModified string
Size int64
Owner Owner
// The class of storage used to store the object.
StorageClass string
}
// Initiator inherit from Owner struct, fields are same
type Initiator Owner
// Owner - bucket owner/principal
type Owner struct {
ID string
DisplayName string
}
// InitiateMultipartUploadResponse container for InitiateMultiPartUpload response, provides uploadID to start MultiPart upload
type InitiateMultipartUploadResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ InitiateMultipartUploadResult" json:"-"`
Bucket string
Key string
UploadID string `xml:"UploadId"`
}
// CompleteMultipartUploadResponse container for completed multipart upload response
type CompleteMultipartUploadResponse struct {
XMLName xml.Name `xml:"http://s3.amazonaws.com/doc/2006-03-01/ CompleteMultipartUploadResult" json:"-"`
Location string
Bucket string
Key string
ETag string
}
// List of not implemented bucket queries
var notimplementedBucketResourceNames = map[string]bool{
"policy": true,
"cors": true,
"lifecycle": true,
"logging": true,
"notification": true,
"replication": true,
"tagging": true,
"versions": true,
"requestPayment": true,
"versioning": true,
"website": true,
}
// List of not implemented object queries
var notimplementedObjectResourceNames = map[string]bool{
"torrent": true,
"acl": true,
"policy": true,
}

View File

@@ -1,268 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"encoding/xml"
"net/http"
)
// APIError structure
type APIError struct {
Code string
Description string
HTTPStatusCode int
}
// APIErrorResponse - error response format
type APIErrorResponse struct {
XMLName xml.Name `xml:"Error" json:"-"`
Code string
Message string
Key string
BucketName string
Resource string
RequestID string `xml:"RequestId"`
HostID string `xml:"HostId"`
}
// Error codes, non exhaustive list - http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html
const (
AccessDenied = iota
BadDigest
BucketAlreadyExists
EntityTooSmall
EntityTooLarge
IncompleteBody
InternalError
InvalidAccessKeyID
InvalidBucketName
InvalidDigest
InvalidRange
InvalidMaxKeys
InvalidMaxUploads
InvalidMaxParts
InvalidPartNumberMarker
MalformedXML
MissingContentLength
MissingRequestBodyError
NoSuchBucket
NoSuchKey
NoSuchUpload
NotImplemented
RequestTimeTooSkewed
SignatureDoesNotMatch
MethodNotAllowed
InvalidPart
InvalidPartOrder
AuthorizationHeaderMalformed
MalformedPOSTRequest
SignatureVersionNotSupported
BucketNotEmpty
RootPathFull
ObjectExistsAsPrefix
)
// APIError code to Error structure map
var errorCodeResponse = map[int]APIError{
InvalidMaxUploads: {
Code: "InvalidArgument",
Description: "Argument maxUploads must be an integer between 0 and 2147483647.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidMaxKeys: {
Code: "InvalidArgument",
Description: "Argument maxKeys must be an integer between 0 and 2147483647.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidMaxParts: {
Code: "InvalidArgument",
Description: "Argument maxParts must be an integer between 1 and 10000.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidPartNumberMarker: {
Code: "InvalidArgument",
Description: "Argument partNumberMarker must be an integer.",
HTTPStatusCode: http.StatusBadRequest,
},
AccessDenied: {
Code: "AccessDenied",
Description: "Access Denied.",
HTTPStatusCode: http.StatusForbidden,
},
BadDigest: {
Code: "BadDigest",
Description: "The Content-MD5 you specified did not match what we received.",
HTTPStatusCode: http.StatusBadRequest,
},
BucketAlreadyExists: {
Code: "BucketAlreadyExists",
Description: "The requested bucket name is not available.",
HTTPStatusCode: http.StatusConflict,
},
EntityTooSmall: {
Code: "EntityTooSmall",
Description: "Your proposed upload is smaller than the minimum allowed object size.",
HTTPStatusCode: http.StatusBadRequest,
},
EntityTooLarge: {
Code: "EntityTooLarge",
Description: "Your proposed upload exceeds the maximum allowed object size.",
HTTPStatusCode: http.StatusBadRequest,
},
IncompleteBody: {
Code: "IncompleteBody",
Description: "You did not provide the number of bytes specified by the Content-Length HTTP header.",
HTTPStatusCode: http.StatusBadRequest,
},
InternalError: {
Code: "InternalError",
Description: "We encountered an internal error, please try again.",
HTTPStatusCode: http.StatusInternalServerError,
},
InvalidAccessKeyID: {
Code: "InvalidAccessKeyID",
Description: "The access key ID you provided does not exist in our records.",
HTTPStatusCode: http.StatusForbidden,
},
InvalidBucketName: {
Code: "InvalidBucketName",
Description: "The specified bucket is not valid.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidDigest: {
Code: "InvalidDigest",
Description: "The Content-MD5 you specified is not valid.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidRange: {
Code: "InvalidRange",
Description: "The requested range cannot be satisfied.",
HTTPStatusCode: http.StatusRequestedRangeNotSatisfiable,
},
MalformedXML: {
Code: "MalformedXML",
Description: "The XML you provided was not well-formed or did not validate against our published schema.",
HTTPStatusCode: http.StatusBadRequest,
},
MissingContentLength: {
Code: "MissingContentLength",
Description: "You must provide the Content-Length HTTP header.",
HTTPStatusCode: http.StatusLengthRequired,
},
MissingRequestBodyError: {
Code: "MissingRequestBodyError",
Description: "Request body is empty.",
HTTPStatusCode: http.StatusLengthRequired,
},
NoSuchBucket: {
Code: "NoSuchBucket",
Description: "The specified bucket does not exist.",
HTTPStatusCode: http.StatusNotFound,
},
NoSuchKey: {
Code: "NoSuchKey",
Description: "The specified key does not exist.",
HTTPStatusCode: http.StatusNotFound,
},
NoSuchUpload: {
Code: "NoSuchUpload",
Description: "The specified multipart upload does not exist.",
HTTPStatusCode: http.StatusNotFound,
},
NotImplemented: {
Code: "NotImplemented",
Description: "A header you provided implies functionality that is not implemented.",
HTTPStatusCode: http.StatusNotImplemented,
},
RequestTimeTooSkewed: {
Code: "RequestTimeTooSkewed",
Description: "The difference between the request time and the server's time is too large.",
HTTPStatusCode: http.StatusForbidden,
},
SignatureDoesNotMatch: {
Code: "SignatureDoesNotMatch",
Description: "The request signature we calculated does not match the signature you provided.",
HTTPStatusCode: http.StatusForbidden,
},
MethodNotAllowed: {
Code: "MethodNotAllowed",
Description: "The specified method is not allowed against this resource.",
HTTPStatusCode: http.StatusMethodNotAllowed,
},
InvalidPart: {
Code: "InvalidPart",
Description: "One or more of the specified parts could not be found.",
HTTPStatusCode: http.StatusBadRequest,
},
InvalidPartOrder: {
Code: "InvalidPartOrder",
Description: "The list of parts was not in ascending order. The parts list must be specified in order by part number.",
HTTPStatusCode: http.StatusBadRequest,
},
AuthorizationHeaderMalformed: {
Code: "AuthorizationHeaderMalformed",
Description: "The authorization header is malformed; the region is wrong; expecting 'us-east-1'.",
HTTPStatusCode: http.StatusBadRequest,
},
MalformedPOSTRequest: {
Code: "MalformedPOSTRequest",
Description: "The body of your POST request is not well-formed multipart/form-data.",
HTTPStatusCode: http.StatusBadRequest,
},
SignatureVersionNotSupported: {
Code: "InvalidRequest",
Description: "The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.",
HTTPStatusCode: http.StatusBadRequest,
},
BucketNotEmpty: {
Code: "BucketNotEmpty",
Description: "The bucket you tried to delete is not empty.",
HTTPStatusCode: http.StatusConflict,
},
RootPathFull: {
Code: "RootPathFull",
Description: "Root path has reached its minimum free disk threshold. Please delete few objects to proceed.",
HTTPStatusCode: http.StatusInternalServerError,
},
ObjectExistsAsPrefix: {
Code: "ObjectExistsAsPrefix",
Description: "An object already exists as your prefix, choose a different prefix to proceed.",
HTTPStatusCode: http.StatusConflict,
},
}
// errorCodeError provides errorCode to Error. It returns empty if the code provided is unknown
func getErrorCode(code int) APIError {
return errorCodeResponse[code]
}
// getErrorResponse gets in standard error and resource value and
// provides a encodable populated response values
func getErrorResponse(err APIError, resource string) APIErrorResponse {
var data = APIErrorResponse{}
data.Code = err.Code
data.Message = err.Description
if resource != "" {
data.Resource = resource
}
// TODO implement this in future
data.RequestID = "3L137"
data.HostID = "3L137"
return data
}

View File

@@ -1,102 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"bytes"
"crypto/rand"
"encoding/xml"
"net/http"
"runtime"
"strconv"
"github.com/minio/minio/pkg/fs"
)
//// helpers
// Static alphaNumeric table used for generating unique request ids
var alphaNumericTable = []byte("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ")
// generateRequestID - Generate request id
func generateRequestID() []byte {
alpha := make([]byte, 16)
rand.Read(alpha)
for i := 0; i < 16; i++ {
alpha[i] = alphaNumericTable[alpha[i]%byte(len(alphaNumericTable))]
}
return alpha
}
// Write http common headers
func setCommonHeaders(w http.ResponseWriter) {
// Set unique request ID for each reply.
w.Header().Set("X-Amz-Request-Id", string(generateRequestID()))
w.Header().Set("Server", ("Minio/" + minioReleaseTag + " (" + runtime.GOOS + "; " + runtime.GOARCH + ")"))
w.Header().Set("Accept-Ranges", "bytes")
}
// Write error response headers
func encodeErrorResponse(response interface{}) []byte {
var bytesBuffer bytes.Buffer
// write common headers
bytesBuffer.WriteString(xml.Header)
e := xml.NewEncoder(&bytesBuffer)
e.Encode(response)
return bytesBuffer.Bytes()
}
// Write object header
func setObjectHeaders(w http.ResponseWriter, metadata fs.ObjectMetadata, contentRange *httpRange) {
// set common headers
if contentRange != nil {
if contentRange.length > 0 {
w.Header().Set("Content-Length", strconv.FormatInt(contentRange.length, 10))
setCommonHeaders(w)
} else {
w.Header().Set("Content-Length", strconv.FormatInt(metadata.Size, 10))
setCommonHeaders(w)
}
} else {
w.Header().Set("Content-Length", strconv.FormatInt(metadata.Size, 10))
setCommonHeaders(w)
}
// set object headers
lastModified := metadata.Created.Format(http.TimeFormat)
// object related headers
w.Header().Set("Content-Type", metadata.ContentType)
if metadata.MD5 != "" {
w.Header().Set("ETag", "\""+metadata.MD5+"\"")
}
w.Header().Set("Last-Modified", lastModified)
// set content range
if contentRange != nil {
if contentRange.start > 0 || contentRange.length > 0 {
w.Header().Set("Content-Range", contentRange.String())
w.WriteHeader(http.StatusPartialContent)
}
}
}
func encodeSuccessResponse(response interface{}) []byte {
var bytesBuffer bytes.Buffer
bytesBuffer.WriteString(xml.Header)
e := xml.NewEncoder(&bytesBuffer)
e.Encode(response)
return bytesBuffer.Bytes()
}

View File

@@ -1,54 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"net/url"
"strconv"
"github.com/minio/minio/pkg/fs"
)
// parse bucket url queries
func getBucketResources(values url.Values) (prefix, marker, delimiter string, maxkeys int, encodingType string) {
prefix = values.Get("prefix")
marker = values.Get("marker")
delimiter = values.Get("delimiter")
maxkeys, _ = strconv.Atoi(values.Get("max-keys"))
encodingType = values.Get("encoding-type")
return
}
// part bucket url queries for ?uploads
func getBucketMultipartResources(values url.Values) (v fs.BucketMultipartResourcesMetadata) {
v.Prefix = values.Get("prefix")
v.KeyMarker = values.Get("key-marker")
v.MaxUploads, _ = strconv.Atoi(values.Get("max-uploads"))
v.Delimiter = values.Get("delimiter")
v.EncodingType = values.Get("encoding-type")
v.UploadIDMarker = values.Get("upload-id-marker")
return
}
// parse object url queries
func getObjectResources(values url.Values) (v fs.ObjectResourcesMetadata) {
v.UploadID = values.Get("uploadId")
v.PartNumberMarker, _ = strconv.Atoi(values.Get("part-number-marker"))
v.MaxParts, _ = strconv.Atoi(values.Get("max-parts"))
v.EncodingType = values.Get("encoding-type")
return
}

View File

@@ -1,246 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"net/http"
"github.com/minio/minio/pkg/fs"
)
// Reply date format
const (
rfcFormat = "2006-01-02T15:04:05.000Z"
)
// takes an array of Bucketmetadata information for serialization
// input:
// array of bucket metadata
//
// output:
// populated struct that can be serialized to match xml and json api spec output
func generateListBucketsResponse(buckets []fs.BucketMetadata) ListBucketsResponse {
var listbuckets []*Bucket
var data = ListBucketsResponse{}
var owner = Owner{}
owner.ID = "minio"
owner.DisplayName = "minio"
for _, bucket := range buckets {
var listbucket = &Bucket{}
listbucket.Name = bucket.Name
listbucket.CreationDate = bucket.Created.Format(rfcFormat)
listbuckets = append(listbuckets, listbucket)
}
data.Owner = owner
data.Buckets.Bucket = listbuckets
return data
}
// generates an AccessControlPolicy response for the said ACL.
func generateAccessControlPolicyResponse(acl fs.BucketACL) AccessControlPolicyResponse {
accessCtrlPolicyResponse := AccessControlPolicyResponse{}
accessCtrlPolicyResponse.Owner = Owner{
ID: "minio",
DisplayName: "minio",
}
defaultGrant := Grant{}
defaultGrant.Grantee.ID = "minio"
defaultGrant.Grantee.DisplayName = "minio"
defaultGrant.Permission = "FULL_CONTROL"
accessCtrlPolicyResponse.AccessControlList.Grant = append(accessCtrlPolicyResponse.AccessControlList.Grant, defaultGrant)
switch {
case acl.IsPublicRead():
publicReadGrant := Grant{}
publicReadGrant.Grantee.ID = "minio"
publicReadGrant.Grantee.DisplayName = "minio"
publicReadGrant.Grantee.URI = "http://acs.amazonaws.com/groups/global/AllUsers"
publicReadGrant.Permission = "READ"
accessCtrlPolicyResponse.AccessControlList.Grant = append(accessCtrlPolicyResponse.AccessControlList.Grant, publicReadGrant)
case acl.IsPublicReadWrite():
publicReadGrant := Grant{}
publicReadGrant.Grantee.ID = "minio"
publicReadGrant.Grantee.DisplayName = "minio"
publicReadGrant.Grantee.URI = "http://acs.amazonaws.com/groups/global/AllUsers"
publicReadGrant.Permission = "READ"
publicReadWriteGrant := Grant{}
publicReadWriteGrant.Grantee.ID = "minio"
publicReadWriteGrant.Grantee.DisplayName = "minio"
publicReadWriteGrant.Grantee.URI = "http://acs.amazonaws.com/groups/global/AllUsers"
publicReadWriteGrant.Permission = "WRITE"
accessCtrlPolicyResponse.AccessControlList.Grant = append(accessCtrlPolicyResponse.AccessControlList.Grant, publicReadGrant)
accessCtrlPolicyResponse.AccessControlList.Grant = append(accessCtrlPolicyResponse.AccessControlList.Grant, publicReadWriteGrant)
}
return accessCtrlPolicyResponse
}
// generates an ListObjects response for the said bucket with other enumerated options.
func generateListObjectsResponse(bucket, prefix, marker, delimiter string, maxKeys int, resp fs.ListObjectsResult) ListObjectsResponse {
var contents []*Object
var prefixes []*CommonPrefix
var owner = Owner{}
var data = ListObjectsResponse{}
owner.ID = "minio"
owner.DisplayName = "minio"
for _, object := range resp.Objects {
var content = &Object{}
if object.Object == "" {
continue
}
content.Key = object.Object
content.LastModified = object.Created.Format(rfcFormat)
if object.MD5 != "" {
content.ETag = "\"" + object.MD5 + "\""
}
content.Size = object.Size
content.StorageClass = "STANDARD"
content.Owner = owner
contents = append(contents, content)
}
// TODO - support EncodingType in xml decoding
data.Name = bucket
data.Contents = contents
data.Prefix = prefix
data.Marker = marker
data.Delimiter = delimiter
data.MaxKeys = maxKeys
data.NextMarker = resp.NextMarker
data.IsTruncated = resp.IsTruncated
for _, prefix := range resp.Prefixes {
var prefixItem = &CommonPrefix{}
prefixItem.Prefix = prefix
prefixes = append(prefixes, prefixItem)
}
data.CommonPrefixes = prefixes
return data
}
// generateInitiateMultipartUploadResponse
func generateInitiateMultipartUploadResponse(bucket, key, uploadID string) InitiateMultipartUploadResponse {
return InitiateMultipartUploadResponse{
Bucket: bucket,
Key: key,
UploadID: uploadID,
}
}
// generateCompleteMultipartUploadResponse
func generateCompleteMultpartUploadResponse(bucket, key, location, etag string) CompleteMultipartUploadResponse {
return CompleteMultipartUploadResponse{
Location: location,
Bucket: bucket,
Key: key,
ETag: etag,
}
}
// generateListPartsResult
func generateListPartsResponse(objectMetadata fs.ObjectResourcesMetadata) ListPartsResponse {
// TODO - support EncodingType in xml decoding
listPartsResponse := ListPartsResponse{}
listPartsResponse.Bucket = objectMetadata.Bucket
listPartsResponse.Key = objectMetadata.Object
listPartsResponse.UploadID = objectMetadata.UploadID
listPartsResponse.StorageClass = "STANDARD"
listPartsResponse.Initiator.ID = "minio"
listPartsResponse.Initiator.DisplayName = "minio"
listPartsResponse.Owner.ID = "minio"
listPartsResponse.Owner.DisplayName = "minio"
listPartsResponse.MaxParts = objectMetadata.MaxParts
listPartsResponse.PartNumberMarker = objectMetadata.PartNumberMarker
listPartsResponse.IsTruncated = objectMetadata.IsTruncated
listPartsResponse.NextPartNumberMarker = objectMetadata.NextPartNumberMarker
listPartsResponse.Part = make([]*Part, len(objectMetadata.Part))
for _, part := range objectMetadata.Part {
newPart := &Part{}
newPart.PartNumber = part.PartNumber
newPart.ETag = "\"" + part.ETag + "\""
newPart.Size = part.Size
newPart.LastModified = part.LastModified.Format(rfcFormat)
listPartsResponse.Part = append(listPartsResponse.Part, newPart)
}
return listPartsResponse
}
// generateListMultipartUploadsResponse
func generateListMultipartUploadsResponse(bucket string, metadata fs.BucketMultipartResourcesMetadata) ListMultipartUploadsResponse {
listMultipartUploadsResponse := ListMultipartUploadsResponse{}
listMultipartUploadsResponse.Bucket = bucket
listMultipartUploadsResponse.Delimiter = metadata.Delimiter
listMultipartUploadsResponse.IsTruncated = metadata.IsTruncated
listMultipartUploadsResponse.EncodingType = metadata.EncodingType
listMultipartUploadsResponse.Prefix = metadata.Prefix
listMultipartUploadsResponse.KeyMarker = metadata.KeyMarker
listMultipartUploadsResponse.NextKeyMarker = metadata.NextKeyMarker
listMultipartUploadsResponse.MaxUploads = metadata.MaxUploads
listMultipartUploadsResponse.NextUploadIDMarker = metadata.NextUploadIDMarker
listMultipartUploadsResponse.UploadIDMarker = metadata.UploadIDMarker
listMultipartUploadsResponse.Upload = make([]*Upload, len(metadata.Upload))
for _, upload := range metadata.Upload {
newUpload := &Upload{}
newUpload.UploadID = upload.UploadID
newUpload.Key = upload.Object
newUpload.Initiated = upload.Initiated.Format(rfcFormat)
listMultipartUploadsResponse.Upload = append(listMultipartUploadsResponse.Upload, newUpload)
}
return listMultipartUploadsResponse
}
// writeSuccessResponse write success headers and response if any.
func writeSuccessResponse(w http.ResponseWriter, response []byte) {
setCommonHeaders(w)
if response == nil {
w.WriteHeader(http.StatusOK)
return
}
w.Write(response)
w.(http.Flusher).Flush()
}
// writeSuccessNoContent write success headers with http status 204
func writeSuccessNoContent(w http.ResponseWriter) {
setCommonHeaders(w)
w.WriteHeader(http.StatusNoContent)
}
// writeErrorRespone write error headers
func writeErrorResponse(w http.ResponseWriter, req *http.Request, errorType int, resource string) {
error := getErrorCode(errorType)
// generate error response
errorResponse := getErrorResponse(error, resource)
encodedErrorResponse := encodeErrorResponse(errorResponse)
// set common headers
setCommonHeaders(w)
// write Header
w.WriteHeader(error.HTTPStatusCode)
// HEAD should have no body, do not attempt to write to it
if req.Method != "HEAD" {
// write error body
w.Write(encodedErrorResponse)
w.(http.Flusher).Flush()
}
}

View File

@@ -1,304 +0,0 @@
/*
* Minio Cloud Storage, (C) 2015 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"bytes"
"encoding/base64"
"io"
"io/ioutil"
"mime/multipart"
"net/http"
"strings"
"time"
"github.com/minio/minio-xl/pkg/probe"
"github.com/minio/minio/pkg/fs"
)
const (
authHeaderPrefix = "AWS4-HMAC-SHA256"
iso8601Format = "20060102T150405Z"
)
// getCredentialsFromAuth parse credentials tag from authorization value
func getCredentialsFromAuth(authValue string) ([]string, *probe.Error) {
if authValue == "" {
return nil, probe.NewError(errMissingAuthHeaderValue)
}
// replace all spaced strings
authValue = strings.Replace(authValue, " ", "", -1)
if !strings.HasPrefix(authValue, authHeaderPrefix) {
return nil, probe.NewError(errMissingFieldsAuthHeader)
}
if !strings.HasPrefix(strings.TrimPrefix(authValue, authHeaderPrefix), "Credential") {
return nil, probe.NewError(errInvalidAuthHeaderPrefix)
}
authValue = strings.TrimPrefix(authValue, authHeaderPrefix)
authFields := strings.Split(strings.TrimSpace(authValue), ",")
if len(authFields) != 3 {
return nil, probe.NewError(errInvalidAuthHeaderValue)
}
credentials := strings.Split(strings.TrimSpace(authFields[0]), "=")
if len(credentials) != 2 {
return nil, probe.NewError(errMissingFieldsCredentialTag)
}
credentialElements := strings.Split(strings.TrimSpace(credentials[1]), "/")
if len(credentialElements) != 5 {
return nil, probe.NewError(errCredentialTagMalformed)
}
return credentialElements, nil
}
func getSignatureFromAuth(authHeaderValue string) (string, *probe.Error) {
authValue := strings.TrimPrefix(authHeaderValue, authHeaderPrefix)
authFields := strings.Split(strings.TrimSpace(authValue), ",")
if len(authFields) != 3 {
return "", probe.NewError(errInvalidAuthHeaderValue)
}
if len(strings.Split(strings.TrimSpace(authFields[2]), "=")) != 2 {
return "", probe.NewError(errMissingFieldsSignatureTag)
}
signature := strings.Split(strings.TrimSpace(authFields[2]), "=")[1]
return signature, nil
}
func getSignedHeadersFromAuth(authHeaderValue string) ([]string, *probe.Error) {
authValue := strings.TrimPrefix(authHeaderValue, authHeaderPrefix)
authFields := strings.Split(strings.TrimSpace(authValue), ",")
if len(authFields) != 3 {
return nil, probe.NewError(errInvalidAuthHeaderValue)
}
if len(strings.Split(strings.TrimSpace(authFields[1]), "=")) != 2 {
return nil, probe.NewError(errMissingFieldsSignedHeadersTag)
}
signedHeaders := strings.Split(strings.Split(strings.TrimSpace(authFields[1]), "=")[1], ";")
return signedHeaders, nil
}
// verify if region value is valid with configured minioRegion.
func isValidRegion(region string, minioRegion string) *probe.Error {
if minioRegion == "" {
minioRegion = "us-east-1"
}
if region != minioRegion && region != "US" {
return probe.NewError(errInvalidRegion)
}
return nil
}
// stripRegion - strip only region from auth header.
func stripRegion(authHeaderValue string) (string, *probe.Error) {
credentialElements, err := getCredentialsFromAuth(authHeaderValue)
if err != nil {
return "", err.Trace(authHeaderValue)
}
region := credentialElements[2]
return region, nil
}
// stripAccessKeyID - strip only access key id from auth header.
func stripAccessKeyID(authHeaderValue string) (string, *probe.Error) {
credentialElements, err := getCredentialsFromAuth(authHeaderValue)
if err != nil {
return "", err.Trace()
}
accessKeyID := credentialElements[0]
if !isValidAccessKey(accessKeyID) {
return "", probe.NewError(errAccessKeyIDInvalid)
}
return accessKeyID, nil
}
// initSignatureV4 initializing signature verification.
func initSignatureV4(req *http.Request) (*fs.Signature, *probe.Error) {
// strip auth from authorization header.
authHeaderValue := req.Header.Get("Authorization")
config, err := loadConfigV2()
if err != nil {
return nil, err.Trace()
}
region, err := stripRegion(authHeaderValue)
if err != nil {
return nil, err.Trace(authHeaderValue)
}
if err = isValidRegion(region, config.Credentials.Region); err != nil {
return nil, err.Trace(authHeaderValue)
}
accessKeyID, err := stripAccessKeyID(authHeaderValue)
if err != nil {
return nil, err.Trace(authHeaderValue)
}
signature, err := getSignatureFromAuth(authHeaderValue)
if err != nil {
return nil, err.Trace(authHeaderValue)
}
signedHeaders, err := getSignedHeadersFromAuth(authHeaderValue)
if err != nil {
return nil, err.Trace(authHeaderValue)
}
if config.Credentials.AccessKeyID == accessKeyID {
signature := &fs.Signature{
AccessKeyID: config.Credentials.AccessKeyID,
SecretAccessKey: config.Credentials.SecretAccessKey,
Region: region,
Signature: signature,
SignedHeaders: signedHeaders,
Request: req,
}
return signature, nil
}
return nil, probe.NewError(errAccessKeyIDInvalid)
}
func extractHTTPFormValues(reader *multipart.Reader) (io.Reader, map[string]string, *probe.Error) {
/// HTML Form values
formValues := make(map[string]string)
filePart := new(bytes.Buffer)
var e error
for e == nil {
var part *multipart.Part
part, e = reader.NextPart()
if part != nil {
if part.FileName() == "" {
buffer, e := ioutil.ReadAll(part)
if e != nil {
return nil, nil, probe.NewError(e)
}
formValues[http.CanonicalHeaderKey(part.FormName())] = string(buffer)
} else {
if _, e := io.Copy(filePart, part); e != nil {
return nil, nil, probe.NewError(e)
}
}
}
}
return filePart, formValues, nil
}
func applyPolicy(formValues map[string]string) *probe.Error {
if formValues["X-Amz-Algorithm"] != "AWS4-HMAC-SHA256" {
return probe.NewError(errUnsupportedAlgorithm)
}
/// Decoding policy
policyBytes, e := base64.StdEncoding.DecodeString(formValues["Policy"])
if e != nil {
return probe.NewError(e)
}
postPolicyForm, err := fs.ParsePostPolicyForm(string(policyBytes))
if err != nil {
return err.Trace()
}
if !postPolicyForm.Expiration.After(time.Now().UTC()) {
return probe.NewError(errPolicyAlreadyExpired)
}
if postPolicyForm.Conditions.Policies["$bucket"].Operator == "eq" {
if formValues["Bucket"] != postPolicyForm.Conditions.Policies["$bucket"].Value {
return probe.NewError(errPolicyMissingFields)
}
}
if postPolicyForm.Conditions.Policies["$x-amz-date"].Operator == "eq" {
if formValues["X-Amz-Date"] != postPolicyForm.Conditions.Policies["$x-amz-date"].Value {
return probe.NewError(errPolicyMissingFields)
}
}
if postPolicyForm.Conditions.Policies["$Content-Type"].Operator == "starts-with" {
if !strings.HasPrefix(formValues["Content-Type"], postPolicyForm.Conditions.Policies["$Content-Type"].Value) {
return probe.NewError(errPolicyMissingFields)
}
}
if postPolicyForm.Conditions.Policies["$Content-Type"].Operator == "eq" {
if formValues["Content-Type"] != postPolicyForm.Conditions.Policies["$Content-Type"].Value {
return probe.NewError(errPolicyMissingFields)
}
}
if postPolicyForm.Conditions.Policies["$key"].Operator == "starts-with" {
if !strings.HasPrefix(formValues["Key"], postPolicyForm.Conditions.Policies["$key"].Value) {
return probe.NewError(errPolicyMissingFields)
}
}
if postPolicyForm.Conditions.Policies["$key"].Operator == "eq" {
if formValues["Key"] != postPolicyForm.Conditions.Policies["$key"].Value {
return probe.NewError(errPolicyMissingFields)
}
}
return nil
}
// initPostPresignedPolicyV4 initializing post policy signature verification
func initPostPresignedPolicyV4(formValues map[string]string) (*fs.Signature, *probe.Error) {
credentialElements := strings.Split(strings.TrimSpace(formValues["X-Amz-Credential"]), "/")
if len(credentialElements) != 5 {
return nil, probe.NewError(errCredentialTagMalformed)
}
accessKeyID := credentialElements[0]
if !isValidAccessKey(accessKeyID) {
return nil, probe.NewError(errAccessKeyIDInvalid)
}
config, err := loadConfigV2()
if err != nil {
return nil, err.Trace()
}
region := credentialElements[2]
if config.Credentials.AccessKeyID == accessKeyID {
signature := &fs.Signature{
AccessKeyID: config.Credentials.AccessKeyID,
SecretAccessKey: config.Credentials.SecretAccessKey,
Region: region,
Signature: formValues["X-Amz-Signature"],
PresignedPolicy: formValues["Policy"],
}
return signature, nil
}
return nil, probe.NewError(errAccessKeyIDInvalid)
}
// initPresignedSignatureV4 initializing presigned signature verification
func initPresignedSignatureV4(req *http.Request) (*fs.Signature, *probe.Error) {
credentialElements := strings.Split(strings.TrimSpace(req.URL.Query().Get("X-Amz-Credential")), "/")
if len(credentialElements) != 5 {
return nil, probe.NewError(errCredentialTagMalformed)
}
accessKeyID := credentialElements[0]
if !isValidAccessKey(accessKeyID) {
return nil, probe.NewError(errAccessKeyIDInvalid)
}
config, err := loadConfigV2()
if err != nil {
return nil, err.Trace()
}
region := credentialElements[2]
signedHeaders := strings.Split(strings.TrimSpace(req.URL.Query().Get("X-Amz-SignedHeaders")), ";")
signature := strings.TrimSpace(req.URL.Query().Get("X-Amz-Signature"))
if config.Credentials.AccessKeyID == accessKeyID {
signature := &fs.Signature{
AccessKeyID: config.Credentials.AccessKeyID,
SecretAccessKey: config.Credentials.SecretAccessKey,
Region: region,
Signature: signature,
SignedHeaders: signedHeaders,
Presigned: true,
Request: req,
}
return signature, nil
}
return nil, probe.NewError(errAccessKeyIDInvalid)
}

View File

@@ -4,37 +4,51 @@ version: "{build}"
# Operating system (build VM template)
os: Windows Server 2012 R2
# Platform.
platform: x64
clone_folder: c:\gopath\src\github.com\minio\minio
# environment variables
# Environment variables
environment:
GOROOT: c:\go17
GOPATH: c:\gopath
GO15VENDOREXPERIMENT: 1
UI_ASSETS: ui-assets.go
UI_ASSETS_ARMOR: ui-assets.asc
# scripts that run after cloning repository
install:
- set PATH=%GOPATH%\bin;c:\go\bin;%PATH%
- set PATH=%GOPATH%\bin;c:\go17\bin;%PATH%
- go version
- go env
- python --version
# to run your custom scripts instead of automatic MSBuild
# To run your custom scripts instead of automatic MSBuild
build_script:
- curl -o ui-assets.go -L https://dl.minio.io/assets/server/%UI_ASSETS%
- curl -o ui-assets.asc -L https://dl.minio.io/assets/server/%UI_ASSETS_ARMOR%
- gpg --batch --no-tty --yes --keyserver pgp.mit.edu --recv-keys F9AAC728
- gpg --batch --no-tty --verify %UI_ASSETS_ARMOR% %UI_ASSETS%
- go test .
- go test -race .
- go test github.com/minio/minio/pkg...
- go test -race github.com/minio/minio/pkg...
# Compile
- appveyor AddCompilationMessage "Starting Compile"
- cd c:\gopath\src\github.com\minio\minio
- go run buildscripts/gen-ldflags.go > temp.txt
- set /p BUILD_LDFLAGS=<temp.txt
- go build -ldflags="%BUILD_LDFLAGS%" -o %GOPATH%\bin\minio.exe
- appveyor AddCompilationMessage "Compile Success"
# to disable automatic tests
test: off
# To run your custom scripts instead of automatic tests
test_script:
# Unit tests
- ps: Add-AppveyorTest "Unit Tests" -Outcome Running
- mkdir build\coverage
- go test -v -timeout 17m -race github.com/minio/minio/cmd...
- go test -v -race github.com/minio/minio/pkg...
- go test -v -coverprofile=build\coverage\coverage.txt -covermode=atomic github.com/minio/minio/cmd
- ps: Update-AppveyorTest "Unit Tests" -Outcome Passed
after_test:
- go tool cover -html=build\coverage\coverage.txt -o build\coverage\coverage.html
- ps: Push-AppveyorArtifact build\coverage\coverage.txt
- ps: Push-AppveyorArtifact build\coverage\coverage.html
# Upload coverage report.
- "SET PATH=C:\\Python34;C:\\Python34\\Scripts;%PATH%"
- pip install codecov
- codecov -X gcov -f "build\coverage\coverage.txt"
# to disable deployment
deploy: off

9
browser/.babelrc Normal file
View File

@@ -0,0 +1,9 @@
{
"presets": [
"es2015",
"react"
],
"plugins": [
"transform-object-rest-spread"
]
}

16
browser/.editorconfig Normal file
View File

@@ -0,0 +1,16 @@
# editorconfig.org
root = true
[*]
indent_style = space
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[*.json]
indent_size = 2
[*.md]
trim_trailing_whitespace = false

23
browser/.esformatter Normal file
View File

@@ -0,0 +1,23 @@
{
"plugins": [
"esformatter-jsx"
],
// Copied from https://github.com/royriojas/esformatter-jsx
"jsx": {
"formatJSX": true, //Duh! that's the default
"attrsOnSameLineAsTag": false, // move each attribute to its own line
"maxAttrsOnTag": 3, // if lower or equal than 3 attributes, they will be kept on a single line
"firstAttributeOnSameLine": true, // keep the first attribute in the same line as the tag
"formatJSXExpressions": true, // default true, if false jsxExpressions won't be recursively formatted
"JSXExpressionsSingleLine": true, // default true, if false the JSXExpressions might span several lines
"alignWithFirstAttribute": false, // do not align attributes with the first tag
"spaceInJSXExpressionContainers": " ", // default to one space. Make it empty if you don't like spaces between JSXExpressionContainers
"removeSpaceBeforeClosingJSX": false, // default false. if true <React.Something /> => <React.Something/>
"closingTagOnNewLine": false, // default false. if true attributes on multiple lines will close the tag on a new line
"JSXAttributeQuotes": "", // possible values "single" or "double". Leave it as empty string if you don't want to modify the attributes' quotes
"htmlOptions": {
// put here the options for js-beautify.html
}
}
}

20
browser/.gitignore vendored Normal file
View File

@@ -0,0 +1,20 @@
**/*.swp
cover.out
*~
minio
!*/
site/
**/*.test
**/*.sublime-workspace
/.idea/
/Minio.iml
**/access.log
build
vendor/**/*.js
vendor/**/*.json
release
.DS_Store
*.syso
coverage.txt
node_modules
production

72
browser/README.md Normal file
View File

@@ -0,0 +1,72 @@
# Minio File Browser
``Minio Browser`` provides minimal set of UI to manage buckets and objects on ``minio`` server. ``Minio Browser`` is written in javascript and released under [Apache 2.0 License](./LICENSE).
## Installation
### Install yarn
```sh
curl -o- -L https://yarnpkg.com/install.sh | bash
yarn
```
### Install `go-bindata` and `go-bindata-assetfs`
If you do not have a working Golang environment, please follow [Install Golang](https://docs.minio.io/docs/how-to-install-golang)
```sh
go get github.com/jteeuwen/go-bindata/...
go get github.com/elazarl/go-bindata-assetfs/...
```
## Generating Assets
### Generate ui-assets.go
```sh
yarn release
```
This generates ui-assets.go in the current directory. Now do `make` in the parent directory to build the minio binary with the newly generated ``ui-assets.go``
### Run Minio Browser with live reload
```sh
yarn dev
```
Open [http://localhost:8080/minio/](http://localhost:8080/minio/) in your browser to play with the application
### Run Minio Browser with live reload on custom port
Edit `browser/webpack.config.js`
```diff
diff --git a/browser/webpack.config.js b/browser/webpack.config.js
index 3ccdaba..9496c56 100644
--- a/browser/webpack.config.js
+++ b/browser/webpack.config.js
@@ -58,6 +58,7 @@ var exports = {
historyApiFallback: {
index: '/minio/'
},
+ port: 8888,
proxy: {
'/minio/webrpc': {
target: 'http://localhost:9000',
@@ -97,7 +98,7 @@ var exports = {
if (process.env.NODE_ENV === 'dev') {
exports.entry = [
'webpack/hot/dev-server',
- 'webpack-dev-server/client?http://localhost:8080',
+ 'webpack-dev-server/client?http://localhost:8888',
path.resolve(__dirname, 'app/index.js')
]
}
```
```sh
yarn dev
```
Open [http://localhost:8888/minio/](http://localhost:8888/minio/) in your browser to play with the application

View File

@@ -0,0 +1,98 @@
.page-load {
position: fixed;
width: 100%;
height: 100%;
top: 0;
left: 0;
background: #32393F;
z-index: 100;
transition: opacity 200ms;
-webkit-transition: opacity 200ms;
}
.pl-0{
opacity: 0;
}
.pl-1 {
display: none;
}
.pl-inner {
position: absolute;
width: 100px;
height: 100px;
left: 50%;
margin-left: -50px;
top: 50%;
margin-top: -50px;
text-align: center;
-webkit-animation: fade-in 500ms;
animation: fade-in 500ms;
-webkit-animation-fill-mode: both;
animation-fill-mode: both;
animation-delay: 350ms;
-webkit-animation-delay: 350ms;
-webkit-backface-visibility: visible;
backface-visibility: visible;
}
.pl-inner:before {
content: '';
position: absolute;
width: 100%;
height: 100%;
left: 0;
top: 0;
display: block;
-webkit-animation: spin 1000ms infinite linear;
animation: spin 1000ms infinite linear;
border: 1px solid rgba(255, 255, 255, 0.2);;
border-left-color: #fff;
border-radius: 50%;
}
.pl-inner > img {
width: 30px;
margin-top: 28px;
}
@-webkit-keyframes fade-in {
0% {
opacity: 0;
}
100% {
opacity: 1;
}
}
@keyframes fade-in {
0% {
opacity: 0;
}
100% {
opacity: 1;
}
}
@-webkit-keyframes spin {
0% {
-webkit-transform: rotate(0deg);
transform: rotate(0deg);
}
100% {
-webkit-transform: rotate(360deg);
transform: rotate(360deg);
}
}
@keyframes spin {
0% {
-webkit-transform: rotate(0deg);
transform: rotate(0deg);
}
100% {
-webkit-transform: rotate(360deg);
transform: rotate(360deg);
}
}

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid meet" viewBox="139.0389584668397 284.78404581828653 12.617622649141168 6.417622649141265"><defs><path d="M139.04 290.7L144.95 284.78L145.46 285.29L139.54 291.2L139.04 290.7Z" id="NsdmgIWbGe"></path><path d="M145.24 285.29L151.15 291.2L151.66 290.7L145.74 284.78L145.24 285.29Z" id="VqPWmhvQEo"></path></defs><g visibility="inherit"><g><use xlink:href="#NsdmgIWbGe" opacity="1" fill="#000000" fill-opacity="1"></use></g><g><use xlink:href="#VqPWmhvQEo" opacity="1" fill="#000000" fill-opacity="1"></use></g></g></svg>

After

Width:  |  Height:  |  Size: 797 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 KiB

BIN
browser/app/img/favicon.ico Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

57
browser/app/img/logo.svg Normal file
View File

@@ -0,0 +1,57 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
viewbox="0 0 160 256"
version="1.1"
id="svg3092"
height="218.14844"
width="137">
<defs
id="defs3094" />
<metadata
id="metadata3097">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<g
transform="translate(0.99999967,-982.85149)"
id="layer1">
<g
transform="matrix(1.0112586,0,0,1.0112586,5.4603732,-13.223714)"
id="g4144">
<g
id="g4140">
<g
style="image-rendering:auto"
id="minio"
transform="matrix(1.0000023,0,0,0.99999799,-739.31646,295.2269)">
<title
id="title3337">Minio Logo</title>
<path
d="m 803.42903,801.80813 c 12.40802,4.17067 27.0499,9.11665 37.95186,12.80906 -12.01295,-21.20683 -27.84305,-34.11687 -37.95186,-40.78578 l 0,27.97672 z m 0,93.72113 -14.22303,8.96217 0,-92.45864 c -1.52985,-0.5139 -2.97948,-0.9981 -4.33405,-1.45259 -19.98593,-6.67189 -32.7207,-17.95703 -35.85168,-31.77904 -2.54577,-11.21386 1.55064,-23.02184 11.24654,-32.39691 8.84929,-8.55225 21.22761,-18.39964 31.17304,-26.31619 3.60329,-2.86658 6.73129,-5.3173 9.2028,-7.39669 2.31406,-1.93977 1.61598,-4.95488 0.57033,-6.21441 -1.74073,-2.09127 -4.61921,-1.74669 -6.56195,-0.32379 -0.10398,0.0802 -5.65595,4.40832 -5.65595,4.40832 l -8.58195,-11.57033 c 0,0 5.60843,-4.14096 5.8223,-4.30137 8.39777,-6.155 19.54034,-4.98758 25.92406,2.71509 3.19039,3.84093 4.68459,8.69779 4.20929,13.67051 -0.47232,4.9549 -2.84579,9.43153 -6.68078,12.5922 -2.58439,2.12988 -5.73912,4.64298 -9.39291,7.54522 -9.70779,7.72641 -21.78905,17.33915 -30.14226,25.41907 -6.1253,5.9233 -8.70671,12.67834 -7.26896,19.02345 1.9873,8.75424 11.33268,16.34105 26.32213,21.37911 l 0,-46.22486 c 40.29563,13.62298 68.76248,61.22321 78.20589,87.64039 0,0 -41.76308,-14.15768 -63.98286,-21.64051 l 0,78.7198"
style="fill:#f8f8f8;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path48" />
<path
d="m 803.42903,826.12513 -14.22303,-4.78261 0,-9.30973 14.22303,4.77667 0,9.31567"
style="fill:#cdccca;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path50" />
<path
d="m 734.75566,745.06155 c -0.23469,0.16636 -0.54956,0.14853 -0.73077,-0.0743 -0.17823,-0.22576 -0.13063,-0.53766 0.0802,-0.73373 4.93113,-4.51525 24.45661,-23.86844 46.30805,-45.2624 l 8.58193,11.57033 c 0,0 -53.54135,34.01288 -54.23942,34.50007"
style="fill:#f14621;fill-opacity:1;fill-rule:nonzero;stroke:none"
id="path52" />
</g>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.0 KiB

View File

@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid meet" width="16" height="4" viewBox="4 10 16 4"><defs><path d="M4 12C4 13.1 4.9 14 6 14C7.1 14 8 13.1 8 12C8 10.9 7.1 10 6 10C4.9 10 4 10.9 4 12ZM16 12C16 13.1 16.9 14 18 14C19.1 14 20 13.1 20 12C20 10.9 19.1 10 18 10C16.9 10 16 10.9 16 12ZM10 12C10 13.1 10.9 14 12 14C13.1 14 14 13.1 14 12C14 10.9 13.1 10 12 10C10.9 10 10 10.9 10 12Z" id="mccsKZxKL3"></path></defs><g visibility="visible"><g><use xlink:href="#mccsKZxKL3" opacity="1" fill="#eaeaea" fill-opacity="1"></use><g><use xlink:href="#mccsKZxKL3" opacity="1" fill-opacity="0" stroke="#000000" stroke-width="1" stroke-opacity="0"></use></g></g></g></svg>

After

Width:  |  Height:  |  Size: 894 B

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path d="M6 10c-1.1 0-2 0.9-2 2s0.9 2 2 2 2-0.9 2-2-0.9-2-2-2zm12 0c-1.1 0-2 0.9-2 2s0.9 2 2 2 2-0.9 2-2-0.9-2-2-2zm-6 0c-1.1 0-2 0.9-2 2s0.9 2 2 2 2-0.9 2-2-0.9-2-2-2z"/></svg>

After

Width:  |  Height:  |  Size: 261 B

View File

@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid meet" width="9" height="9" viewBox="326.76441742035513 536.0133077721175 13 13"><defs><path d="M339.76 536.01L326.76 549.01L339.76 549.01L339.76 536.01Z" id="kt3PSf43ua"></path></defs><g visibility="visible"><g><use xlink:href="#kt3PSf43ua" opacity="1" fill="#dadada" fill-opacity="1"></use></g></g></svg>

After

Width:  |  Height:  |  Size: 586 B

56
browser/app/index.html Normal file
View File

@@ -0,0 +1,56 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Minio Browser</title>
<link rel="stylesheet" href="/minio/loader.css" type="text/css">
</head>
<body>
<div class="page-load">
<div class="pl-inner">
<img src="/minio/logo.svg" alt="">
</div>
</div>
<div id="root"></div>
<!--[if lt IE 11]>
<div class="ie-warning">
<div class="iw-inner">
<i class="iwi-icon fa fa-warning"></i>
You are using Internet Explorer version 12.0 or lower. Due to security issues and lack of support for Web Standards it is highly recommended that you upgrade to a modern browser
<ul>
<li>
<a href="http://www.google.com/chrome/">
<img src="/minio/chrome.png" alt="">
<div>Chrome</div>
</a>
</li>
<li>
<a href="https://www.mozilla.org/en-US/firefox/new/">
<img src="/minio/firefox.png" alt="">
<div>Firefox</div>
</a>
</li>
<li>
<a href="https://www.apple.com/safari/">
<img src="/minio/safari.png" alt="">
<div>Safari</div>
</a>
</li>
</ul>
<div class="iwi-skip">Skip & Continue</div>
</div>
</div>
<![endif]-->
<script>currentUiVersion = 'MINIO_UI_VERSION'</script>
<script src="/minio/index_bundle.js"></script>
</body>
</html>

116
browser/app/index.js Normal file
View File

@@ -0,0 +1,116 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import 'babel-polyfill'
import './less/main.less'
import React from 'react'
import ReactDOM from 'react-dom'
import thunkMiddleware from 'redux-thunk'
import createStore from 'redux/lib/createStore'
import applyMiddleware from 'redux/lib/applyMiddleware'
import Route from 'react-router/lib/Route'
import Router from 'react-router/lib/Router'
import browserHistory from 'react-router/lib/browserHistory'
import IndexRoute from 'react-router/lib/IndexRoute'
import Provider from 'react-redux/lib/components/Provider'
import connect from 'react-redux/lib/components/connect'
import Moment from 'moment'
import { minioBrowserPrefix } from './js/constants.js'
import * as actions from './js/actions.js'
import reducer from './js/reducers.js'
import _Login from './js/components/Login.js'
import _Browse from './js/components/Browse.js'
import fontAwesome from 'font-awesome/css/font-awesome.css'
import Web from './js/web'
window.Web = Web
import storage from 'local-storage-fallback'
const store = applyMiddleware(thunkMiddleware)(createStore)(reducer)
const Browse = connect(state => state)(_Browse)
const Login = connect(state => state)(_Login)
let web = new Web(`${window.location.protocol}//${window.location.host}${minioBrowserPrefix}/webrpc`, store.dispatch)
window.web = web
store.dispatch(actions.setWeb(web))
function authNeeded(nextState, replace, cb) {
if (web.LoggedIn()) {
return cb()
}
if (location.pathname === minioBrowserPrefix || location.pathname === minioBrowserPrefix + '/') {
replace(`${minioBrowserPrefix}/login`)
}
return cb()
}
function authNotNeeded(nextState, replace) {
if (web.LoggedIn()) {
replace(`${minioBrowserPrefix}`)
}
}
const App = (props) => {
return <div>
{ props.children }
</div>
}
ReactDOM.render((
<Provider store={ store } web={ web }>
<Router history={ browserHistory }>
<Route path='/' component={ App }>
<Route path='minio' component={ App }>
<IndexRoute component={ Browse } onEnter={ authNeeded } />
<Route path='login' component={ Login } onEnter={ authNotNeeded } />
<Route path=':bucket' component={ Browse } onEnter={ authNeeded } />
<Route path=':bucket/*' component={ Browse } onEnter={ authNeeded } />
</Route>
</Route>
</Router>
</Provider>
), document.getElementById('root'))
//Page loader
let delay = [0, 400]
let i = 0
function handleLoader() {
if (i < 2) {
setTimeout(function() {
document.querySelector('.page-load').classList.add('pl-' + i)
i++
handleLoader()
}, delay[i])
}
}
handleLoader()
if (storage.getItem('newlyUpdated')) {
store.dispatch(actions.showAlert({
type: 'success',
message: "Updated to the latest UI Version."
}))
storage.removeItem('newlyUpdated')
}

View File

@@ -0,0 +1,43 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import expect from 'expect';
import JSONrpc from '../jsonrpc';
describe('jsonrpc', () => {
it('should fail with invalid endpoint', (done) => {
try {
let jsonRPC = new JSONrpc({
endpoint: 'htt://localhost:9000',
namespace: 'Test'
});
} catch (e) {
done();
}
});
it('should succeed with valid endpoint', () => {
let jsonRPC = new JSONrpc({
endpoint: 'http://localhost:9000/webrpc',
namespace: 'Test'
});
expect(jsonRPC.version).toEqual('2.0');
expect(jsonRPC.host).toEqual('localhost');
expect(jsonRPC.port).toEqual('9000');
expect(jsonRPC.path).toEqual('/webrpc');
expect(jsonRPC.scheme).toEqual('http');
});
});

637
browser/app/js/actions.js Normal file
View File

@@ -0,0 +1,637 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import Moment from 'moment'
import browserHistory from 'react-router/lib/browserHistory'
import storage from 'local-storage-fallback'
import { minioBrowserPrefix } from './constants'
export const SET_WEB = 'SET_WEB'
export const SET_CURRENT_BUCKET = 'SET_CURRENT_BUCKET'
export const SET_CURRENT_PATH = 'SET_CURRENT_PATH'
export const SET_BUCKETS = 'SET_BUCKETS'
export const ADD_BUCKET = 'ADD_BUCKET'
export const SET_VISIBLE_BUCKETS = 'SET_VISIBLE_BUCKETS'
export const SET_OBJECTS = 'SET_OBJECTS'
export const APPEND_OBJECTS = 'APPEND_OBJECTS'
export const RESET_OBJECTS = 'RESET_OBJECTS'
export const SET_STORAGE_INFO = 'SET_STORAGE_INFO'
export const SET_SERVER_INFO = 'SET_SERVER_INFO'
export const SHOW_MAKEBUCKET_MODAL = 'SHOW_MAKEBUCKET_MODAL'
export const ADD_UPLOAD = 'ADD_UPLOAD'
export const STOP_UPLOAD = 'STOP_UPLOAD'
export const UPLOAD_PROGRESS = 'UPLOAD_PROGRESS'
export const SET_ALERT = 'SET_ALERT'
export const SET_LOGIN_ERROR = 'SET_LOGIN_ERROR'
export const SET_SHOW_ABORT_MODAL = 'SET_SHOW_ABORT_MODAL'
export const SHOW_ABOUT = 'SHOW_ABOUT'
export const SET_SORT_NAME_ORDER = 'SET_SORT_NAME_ORDER'
export const SET_SORT_SIZE_ORDER = 'SET_SORT_SIZE_ORDER'
export const SET_SORT_DATE_ORDER = 'SET_SORT_DATE_ORDER'
export const SET_LATEST_UI_VERSION = 'SET_LATEST_UI_VERSION'
export const SET_SIDEBAR_STATUS = 'SET_SIDEBAR_STATUS'
export const SET_LOGIN_REDIRECT_PATH = 'SET_LOGIN_REDIRECT_PATH'
export const SET_LOAD_BUCKET = 'SET_LOAD_BUCKET'
export const SET_LOAD_PATH = 'SET_LOAD_PATH'
export const SHOW_SETTINGS = 'SHOW_SETTINGS'
export const SET_SETTINGS = 'SET_SETTINGS'
export const SHOW_BUCKET_POLICY = 'SHOW_BUCKET_POLICY'
export const SET_POLICIES = 'SET_POLICIES'
export const SET_SHARE_OBJECT = 'SET_SHARE_OBJECT'
export const DELETE_CONFIRMATION = 'DELETE_CONFIRMATION'
export const SET_PREFIX_WRITABLE = 'SET_PREFIX_WRITABLE'
export const REMOVE_OBJECT = 'REMOVE_OBJECT'
export const CHECKED_OBJECTS_ADD = 'CHECKED_OBJECTS_ADD'
export const CHECKED_OBJECTS_REMOVE = 'CHECKED_OBJECTS_REMOVE'
export const CHECKED_OBJECTS_RESET = 'CHECKED_OBJECTS_RESET'
export const showDeleteConfirmation = (object) => {
return {
type: DELETE_CONFIRMATION,
payload: {
object,
show: true
}
}
}
export const hideDeleteConfirmation = () => {
return {
type: DELETE_CONFIRMATION,
payload: {
object: '',
show: false
}
}
}
export const showShareObject = (object, url) => {
return {
type: SET_SHARE_OBJECT,
shareObject: {
object,
url,
show: true
}
}
}
export const hideShareObject = () => {
return {
type: SET_SHARE_OBJECT,
shareObject: {
url: '',
show: false
}
}
}
export const shareObject = (object, days, hours, minutes) => (dispatch, getState) => {
const {currentBucket, web} = getState()
let host = location.host
let bucket = currentBucket
if (!web.LoggedIn()) {
dispatch(showShareObject(object, `${host}/${bucket}/${object}`))
return
}
let expiry = days * 24 * 60 * 60 + hours * 60 * 60 + minutes * 60
web.PresignedGet({
host,
bucket,
object,
expiry
})
.then(obj => {
dispatch(showShareObject(object, obj.url))
dispatch(showAlert({
type: 'success',
message: `Object shared. Expires in ${days} days ${hours} hours ${minutes} minutes.`
}))
})
.catch(err => {
dispatch(showAlert({
type: 'danger',
message: err.message
}))
})
}
export const setLoginRedirectPath = (path) => {
return {
type: SET_LOGIN_REDIRECT_PATH,
path
}
}
export const setLoadPath = (loadPath) => {
return {
type: SET_LOAD_PATH,
loadPath
}
}
export const setLoadBucket = (loadBucket) => {
return {
type: SET_LOAD_BUCKET,
loadBucket
}
}
export const setWeb = web => {
return {
type: SET_WEB,
web
}
}
export const setBuckets = buckets => {
return {
type: SET_BUCKETS,
buckets
}
}
export const addBucket = bucket => {
return {
type: ADD_BUCKET,
bucket
}
}
export const showMakeBucketModal = () => {
return {
type: SHOW_MAKEBUCKET_MODAL,
showMakeBucketModal: true
}
}
export const hideAlert = () => {
return {
type: SET_ALERT,
alert: {
show: false,
message: '',
type: ''
}
}
}
export const showAlert = alert => {
return (dispatch, getState) => {
let alertTimeout = null
if (alert.type !== 'danger') {
alertTimeout = setTimeout(() => {
dispatch({
type: SET_ALERT,
alert: {
show: false
}
})
}, 5000)
}
dispatch({
type: SET_ALERT,
alert: Object.assign({}, alert, {
show: true,
alertTimeout
})
})
}
}
export const removeObject = object => {
return {
type: REMOVE_OBJECT,
object
}
}
export const setSidebarStatus = (status) => {
return {
type: SET_SIDEBAR_STATUS,
sidebarStatus: status
}
}
export const hideMakeBucketModal = () => {
return {
type: SHOW_MAKEBUCKET_MODAL,
showMakeBucketModal: false
}
}
export const setVisibleBuckets = visibleBuckets => {
return {
type: SET_VISIBLE_BUCKETS,
visibleBuckets
}
}
const appendObjects = (objects, marker, istruncated) => {
return {
type: APPEND_OBJECTS,
objects,
marker,
istruncated
}
}
export const setObjects = (objects) => {
return {
type: SET_OBJECTS,
objects,
}
}
export const resetObjects = () => {
return {
type: RESET_OBJECTS
}
}
export const setCurrentBucket = currentBucket => {
return {
type: SET_CURRENT_BUCKET,
currentBucket
}
}
export const setCurrentPath = currentPath => {
return {
type: SET_CURRENT_PATH,
currentPath
}
}
export const setStorageInfo = storageInfo => {
return {
type: SET_STORAGE_INFO,
storageInfo
}
}
export const setServerInfo = serverInfo => {
return {
type: SET_SERVER_INFO,
serverInfo
}
}
const setPrefixWritable = prefixWritable => {
return {
type: SET_PREFIX_WRITABLE,
prefixWritable,
}
}
export const selectBucket = (newCurrentBucket, prefix) => {
if (!prefix)
prefix = ''
return (dispatch, getState) => {
let web = getState().web
let currentBucket = getState().currentBucket
if (currentBucket !== newCurrentBucket) dispatch(setLoadBucket(newCurrentBucket))
dispatch(setCurrentBucket(newCurrentBucket))
dispatch(selectPrefix(prefix))
return
}
}
export const listObjects = () => {
return (dispatch, getState) => {
const {currentBucket, currentPath, marker, objects, istruncated, web} = getState()
if (!istruncated) return
web.ListObjects({
bucketName: currentBucket,
prefix: currentPath,
marker: marker
})
.then(res => {
let objects = res.objects
if (!objects)
objects = []
objects = objects.map(object => {
object.name = object.name.replace(`${currentPath}`, '');
return object
})
dispatch(appendObjects(objects, res.nextmarker, res.istruncated))
dispatch(setPrefixWritable(res.writable))
dispatch(setLoadBucket(''))
dispatch(setLoadPath(''))
})
.catch(err => {
dispatch(showAlert({
type: 'danger',
message: err.message
}))
dispatch(setLoadBucket(''))
dispatch(setLoadPath(''))
// Use browserHistory.replace instead of push so that browser back button works fine.
browserHistory.replace(`${minioBrowserPrefix}/login`)
})
}
}
export const selectPrefix = prefix => {
return (dispatch, getState) => {
const {currentBucket, web} = getState()
dispatch(resetObjects())
dispatch(setLoadPath(prefix))
web.ListObjects({
bucketName: currentBucket,
prefix,
marker: ""
})
.then(res => {
let objects = res.objects
if (!objects)
objects = []
objects = objects.map(object => {
object.name = object.name.replace(`${prefix}`, '');
return object
})
dispatch(appendObjects(
objects,
res.nextmarker,
res.istruncated
))
dispatch(setPrefixWritable(res.writable))
dispatch(setSortNameOrder(false))
dispatch(setCurrentPath(prefix))
dispatch(setLoadBucket(''))
dispatch(setLoadPath(''))
})
.catch(err => {
dispatch(showAlert({
type: 'danger',
message: err.message
}))
dispatch(setLoadBucket(''))
dispatch(setLoadPath(''))
// Use browserHistory.replace instead of push so that browser back button works fine.
browserHistory.replace(`${minioBrowserPrefix}/login`)
})
}
}
export const addUpload = options => {
return {
type: ADD_UPLOAD,
slug: options.slug,
size: options.size,
xhr: options.xhr,
name: options.name
}
}
export const stopUpload = options => {
return {
type: STOP_UPLOAD,
slug: options.slug
}
}
export const uploadProgress = options => {
return {
type: UPLOAD_PROGRESS,
slug: options.slug,
loaded: options.loaded
}
}
export const setShowAbortModal = showAbortModal => {
return {
type: SET_SHOW_ABORT_MODAL,
showAbortModal
}
}
export const setLoginError = () => {
return {
type: SET_LOGIN_ERROR,
loginError: true
}
}
export const downloadSelected = (url, req, xhr) => {
return (dispatch) => {
var anchor = document.createElement('a')
document.body.appendChild(anchor);
xhr.open('POST', url, true)
xhr.responseType = 'blob'
xhr.onload = function(e) {
if (this.status == 200) {
dispatch(checkedObjectsReset())
var blob = new Blob([this.response], {
type: 'octet/stream'
})
var blobUrl = window.URL.createObjectURL(blob);
var separator = req.prefix.length > 1 ? '-' : ''
anchor.href = blobUrl
anchor.download = req.bucketName+separator+req.prefix.slice(0, -1)+'.zip';
anchor.click()
window.URL.revokeObjectURL(blobUrl)
anchor.remove()
}
};
xhr.send(JSON.stringify(req));
}
}
export const uploadFile = (file, xhr) => {
return (dispatch, getState) => {
const {currentBucket, currentPath} = getState()
const objectName = `${currentPath}${file.name}`
const uploadUrl = `${window.location.origin}/minio/upload/${currentBucket}/${objectName}`
// The slug is a unique identifer for the file upload.
const slug = `${currentBucket}-${currentPath}-${file.name}`
xhr.open('PUT', uploadUrl, true)
xhr.withCredentials = false
const token = storage.getItem('token')
if (token) xhr.setRequestHeader("Authorization", 'Bearer ' + storage.getItem('token'))
xhr.setRequestHeader('x-amz-date', Moment().utc().format('YYYYMMDDTHHmmss') + 'Z')
dispatch(addUpload({
slug,
xhr,
size: file.size,
name: file.name
}))
xhr.onload = function(event) {
if (xhr.status == 401 || xhr.status == 403 || xhr.status == 500) {
setShowAbortModal(false)
dispatch(stopUpload({
slug
}))
dispatch(showAlert({
type: 'danger',
message: 'Unauthorized request.'
}))
}
if (xhr.status == 200) {
setShowAbortModal(false)
dispatch(stopUpload({
slug
}))
dispatch(showAlert({
type: 'success',
message: 'File \'' + file.name + '\' uploaded successfully.'
}))
dispatch(selectPrefix(currentPath))
}
}
xhr.upload.addEventListener('error', event => {
dispatch(showAlert({
type: 'danger',
message: 'Error occurred uploading \'' + file.name + '\'.'
}))
dispatch(stopUpload({
slug
}))
})
xhr.upload.addEventListener('progress', event => {
if (event.lengthComputable) {
let loaded = event.loaded
let total = event.total
// Update the counter.
dispatch(uploadProgress({
slug,
loaded
}))
}
})
xhr.send(file)
}
}
export const showAbout = () => {
return {
type: SHOW_ABOUT,
showAbout: true
}
}
export const hideAbout = () => {
return {
type: SHOW_ABOUT,
showAbout: false
}
}
export const setSortNameOrder = (sortNameOrder) => {
return {
type: SET_SORT_NAME_ORDER,
sortNameOrder
}
}
export const setSortSizeOrder = (sortSizeOrder) => {
return {
type: SET_SORT_SIZE_ORDER,
sortSizeOrder
}
}
export const setSortDateOrder = (sortDateOrder) => {
return {
type: SET_SORT_DATE_ORDER,
sortDateOrder
}
}
export const setLatestUIVersion = (latestUiVersion) => {
return {
type: SET_LATEST_UI_VERSION,
latestUiVersion
}
}
export const showSettings = () => {
return {
type: SHOW_SETTINGS,
showSettings: true
}
}
export const hideSettings = () => {
return {
type: SHOW_SETTINGS,
showSettings: false
}
}
export const setSettings = (settings) => {
return {
type: SET_SETTINGS,
settings
}
}
export const showBucketPolicy = () => {
return {
type: SHOW_BUCKET_POLICY,
showBucketPolicy: true
}
}
export const hideBucketPolicy = () => {
return {
type: SHOW_BUCKET_POLICY,
showBucketPolicy: false
}
}
export const setPolicies = (policies) => {
return {
type: SET_POLICIES,
policies
}
}
export const checkedObjectsAdd = (objectName) => {
return {
type: CHECKED_OBJECTS_ADD,
objectName
}
}
export const checkedObjectsRemove = (objectName) => {
return {
type: CHECKED_OBJECTS_REMOVE,
objectName
}
}
export const checkedObjectsReset = (objectName) => {
return {
type: CHECKED_OBJECTS_RESET,
objectName
}
}

View File

@@ -0,0 +1,800 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import classNames from 'classnames'
import browserHistory from 'react-router/lib/browserHistory'
import humanize from 'humanize'
import Moment from 'moment'
import Modal from 'react-bootstrap/lib/Modal'
import ModalBody from 'react-bootstrap/lib/ModalBody'
import ModalHeader from 'react-bootstrap/lib/ModalHeader'
import Alert from 'react-bootstrap/lib/Alert'
import OverlayTrigger from 'react-bootstrap/lib/OverlayTrigger'
import Tooltip from 'react-bootstrap/lib/Tooltip'
import Dropdown from 'react-bootstrap/lib/Dropdown'
import MenuItem from 'react-bootstrap/lib/MenuItem'
import InputGroup from '../components/InputGroup'
import Dropzone from '../components/Dropzone'
import ObjectsList from '../components/ObjectsList'
import SideBar from '../components/SideBar'
import Path from '../components/Path'
import BrowserUpdate from '../components/BrowserUpdate'
import UploadModal from '../components/UploadModal'
import SettingsModal from '../components/SettingsModal'
import PolicyInput from '../components/PolicyInput'
import Policy from '../components/Policy'
import BrowserDropdown from '../components/BrowserDropdown'
import ConfirmModal from './ConfirmModal'
import logo from '../../img/logo.svg'
import * as actions from '../actions'
import * as utils from '../utils'
import * as mime from '../mime'
import { minioBrowserPrefix } from '../constants'
import CopyToClipboard from 'react-copy-to-clipboard'
import storage from 'local-storage-fallback'
import InfiniteScroll from 'react-infinite-scroller';
export default class Browse extends React.Component {
componentDidMount() {
const {web, dispatch, currentBucket} = this.props
if (!web.LoggedIn()) return
web.StorageInfo()
.then(res => {
let storageInfo = Object.assign({}, {
total: res.storageInfo.Total,
free: res.storageInfo.Free
})
storageInfo.used = storageInfo.total - storageInfo.free
dispatch(actions.setStorageInfo(storageInfo))
return web.ServerInfo()
})
.then(res => {
let serverInfo = Object.assign({}, {
version: res.MinioVersion,
memory: res.MinioMemory,
platform: res.MinioPlatform,
runtime: res.MinioRuntime,
info: res.MinioGlobalInfo
})
dispatch(actions.setServerInfo(serverInfo))
})
.catch(err => {
dispatch(actions.showAlert({
type: 'danger',
message: err.message
}))
})
}
componentWillMount() {
const {dispatch} = this.props
// Clear out any stale message in the alert of Login page
dispatch(actions.showAlert({
type: 'danger',
message: ''
}))
if (web.LoggedIn()) {
web.ListBuckets()
.then(res => {
let buckets
if (!res.buckets)
buckets = []
else
buckets = res.buckets.map(bucket => bucket.name)
if (buckets.length) {
dispatch(actions.setBuckets(buckets))
dispatch(actions.setVisibleBuckets(buckets))
if (location.pathname === minioBrowserPrefix || location.pathname === minioBrowserPrefix + '/') {
browserHistory.push(utils.pathJoin(buckets[0]))
}
}
})
}
this.history = browserHistory.listen(({pathname}) => {
let decPathname = decodeURI(pathname)
if (decPathname === `${minioBrowserPrefix}/login`) return // FIXME: better organize routes and remove this
if (!decPathname.endsWith('/'))
decPathname += '/'
if (decPathname === minioBrowserPrefix + '/') {
return
}
let obj = utils.pathSlice(decPathname)
if (!web.LoggedIn()) {
dispatch(actions.setBuckets([obj.bucket]))
dispatch(actions.setVisibleBuckets([obj.bucket]))
}
dispatch(actions.selectBucket(obj.bucket, obj.prefix))
})
}
componentWillUnmount() {
this.history()
}
selectBucket(e, bucket) {
e.preventDefault()
if (bucket === this.props.currentBucket) return
browserHistory.push(utils.pathJoin(bucket))
}
searchBuckets(e) {
e.preventDefault()
let {buckets} = this.props
this.props.dispatch(actions.setVisibleBuckets(buckets.filter(bucket => bucket.indexOf(e.target.value) > -1)))
}
listObjects() {
const {dispatch} = this.props
dispatch(actions.listObjects())
}
selectPrefix(e, prefix) {
e.preventDefault()
const {dispatch, currentPath, web, currentBucket} = this.props
const encPrefix = encodeURI(prefix)
if (prefix.endsWith('/') || prefix === '') {
if (prefix === currentPath) return
browserHistory.push(utils.pathJoin(currentBucket, encPrefix))
} else {
window.location = `${window.location.origin}/minio/download/${currentBucket}/${encPrefix}?token=${storage.getItem('token')}`
}
}
makeBucket(e) {
e.preventDefault()
const bucketName = this.refs.makeBucketRef.value
this.refs.makeBucketRef.value = ''
const {web, dispatch} = this.props
this.hideMakeBucketModal()
web.MakeBucket({
bucketName
})
.then(() => {
dispatch(actions.addBucket(bucketName))
dispatch(actions.selectBucket(bucketName))
})
.catch(err => dispatch(actions.showAlert({
type: 'danger',
message: err.message
})))
}
hideMakeBucketModal() {
const {dispatch} = this.props
dispatch(actions.hideMakeBucketModal())
}
showMakeBucketModal(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.showMakeBucketModal())
}
showAbout(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.showAbout())
}
hideAbout(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.hideAbout())
}
showBucketPolicy(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.showBucketPolicy())
}
hideBucketPolicy(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.hideBucketPolicy())
}
uploadFile(e) {
e.preventDefault()
const {dispatch, buckets} = this.props
if (buckets.length === 0) {
dispatch(actions.showAlert({
type: 'danger',
message: "Bucket needs to be created before trying to upload files."
}))
return
}
let file = e.target.files[0]
e.target.value = null
this.xhr = new XMLHttpRequest()
dispatch(actions.uploadFile(file, this.xhr))
}
removeObject() {
const {web, dispatch, currentPath, currentBucket, deleteConfirmation, checkedObjects} = this.props
let objects = []
if (checkedObjects.length > 0) {
objects = checkedObjects.map(obj => `${currentPath}${obj}`)
} else {
objects = [deleteConfirmation.object]
}
web.RemoveObject({
bucketname: currentBucket,
objects: objects
})
.then(() => {
this.hideDeleteConfirmation()
if (checkedObjects.length > 0) {
for (let i = 0; i < checkedObjects.length; i++) {
dispatch(actions.removeObject(checkedObjects[i].replace(currentPath, '')))
}
dispatch(actions.checkedObjectsReset())
} else {
let delObject = deleteConfirmation.object.replace(currentPath, '')
dispatch(actions.removeObject(delObject))
}
})
.catch(e => dispatch(actions.showAlert({
type: 'danger',
message: e.message
})))
}
hideAlert(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.hideAlert())
}
showDeleteConfirmation(e, object) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.showDeleteConfirmation(object))
}
hideDeleteConfirmation() {
const {dispatch} = this.props
dispatch(actions.hideDeleteConfirmation())
}
shareObject(e, object) {
e.preventDefault()
const {dispatch} = this.props
// let expiry = 5 * 24 * 60 * 60 // 5 days expiry by default
dispatch(actions.shareObject(object, 5, 0, 0))
}
hideShareObjectModal() {
const {dispatch} = this.props
dispatch(actions.hideShareObject())
}
dataType(name, contentType) {
return mime.getDataType(name, contentType)
}
sortObjectsByName(e) {
const {dispatch, objects, sortNameOrder} = this.props
dispatch(actions.setObjects(utils.sortObjectsByName(objects, !sortNameOrder)))
dispatch(actions.setSortNameOrder(!sortNameOrder))
}
sortObjectsBySize() {
const {dispatch, objects, sortSizeOrder} = this.props
dispatch(actions.setObjects(utils.sortObjectsBySize(objects, !sortSizeOrder)))
dispatch(actions.setSortSizeOrder(!sortSizeOrder))
}
sortObjectsByDate() {
const {dispatch, objects, sortDateOrder} = this.props
dispatch(actions.setObjects(utils.sortObjectsByDate(objects, !sortDateOrder)))
dispatch(actions.setSortDateOrder(!sortDateOrder))
}
logout(e) {
const {web} = this.props
e.preventDefault()
web.Logout()
browserHistory.push(`${minioBrowserPrefix}/login`)
}
fullScreen(e) {
e.preventDefault()
let el = document.documentElement
if (el.requestFullscreen) {
el.requestFullscreen()
}
if (el.mozRequestFullScreen) {
el.mozRequestFullScreen()
}
if (el.webkitRequestFullscreen) {
el.webkitRequestFullscreen()
}
if (el.msRequestFullscreen) {
el.msRequestFullscreen()
}
}
toggleSidebar(status) {
this.props.dispatch(actions.setSidebarStatus(status))
}
hideSidebar(event) {
let e = event || window.event;
// Support all browsers.
let target = e.srcElement || e.target;
if (target.nodeType === 3) // Safari support.
target = target.parentNode;
let targetID = target.id;
if (!(targetID === 'feh-trigger')) {
this.props.dispatch(actions.setSidebarStatus(false))
}
}
showSettings(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.showSettings())
}
showMessage() {
const {dispatch} = this.props
dispatch(actions.showAlert({
type: 'success',
message: 'Link copied to clipboard!'
}))
this.hideShareObjectModal()
}
selectTexts() {
this.refs.copyTextInput.select()
}
handleExpireValue(targetInput, inc, object) {
inc === -1 ? this.refs[targetInput].stepDown(1) : this.refs[targetInput].stepUp(1)
if (this.refs.expireDays.value == 7) {
this.refs.expireHours.value = 0
this.refs.expireMins.value = 0
}
if (this.refs.expireDays.value + this.refs.expireHours.value + this.refs.expireMins.value == 0) {
this.refs.expireDays.value = 7
}
const {dispatch} = this.props
dispatch(actions.shareObject(object, this.refs.expireDays.value, this.refs.expireHours.value, this.refs.expireMins.value))
}
checkObject(e, objectName) {
const {dispatch} = this.props
e.target.checked ? dispatch(actions.checkedObjectsAdd(objectName)) : dispatch(actions.checkedObjectsRemove(objectName))
}
downloadSelected() {
const {dispatch} = this.props
let req = {
bucketName: this.props.currentBucket,
objects: this.props.checkedObjects,
prefix: this.props.currentPath
}
let requestUrl = location.origin + "/minio/zip?token=" + localStorage.token
this.xhr = new XMLHttpRequest()
dispatch(actions.downloadSelected(requestUrl, req, this.xhr))
}
clearSelected() {
const {dispatch} = this.props
dispatch(actions.checkedObjectsReset())
}
render() {
const {total, free} = this.props.storageInfo
const {showMakeBucketModal, alert, sortNameOrder, sortSizeOrder, sortDateOrder, showAbout, showBucketPolicy, checkedObjects} = this.props
const {version, memory, platform, runtime} = this.props.serverInfo
const {sidebarStatus} = this.props
const {showSettings} = this.props
const {policies, currentBucket, currentPath} = this.props
const {deleteConfirmation} = this.props
const {shareObject} = this.props
const {web, prefixWritable, istruncated} = this.props
// Don't always show the SettingsModal. This is done here instead of in
// SettingsModal.js so as to allow for #componentWillMount to handle
// the loading of the settings.
let settingsModal = showSettings ? <SettingsModal /> : <noscript></noscript>
let alertBox = <Alert className={ classNames({
'alert': true,
'animated': true,
'fadeInDown': alert.show,
'fadeOutUp': !alert.show
}) } bsStyle={ alert.type } onDismiss={ this.hideAlert.bind(this) }>
<div className='text-center'>
{ alert.message }
</div>
</Alert>
// Make sure you don't show a fading out alert box on the initial web-page load.
if (!alert.message)
alertBox = ''
let signoutTooltip = <Tooltip id="tt-sign-out">
Sign out
</Tooltip>
let uploadTooltip = <Tooltip id="tt-upload-file">
Upload file
</Tooltip>
let makeBucketTooltip = <Tooltip id="tt-create-bucket">
Create bucket
</Tooltip>
let loginButton = ''
let browserDropdownButton = ''
let storageUsageDetails = ''
let used = total - free
let usedPercent = (used / total) * 100 + '%'
let freePercent = free * 100 / total
if (web.LoggedIn()) {
browserDropdownButton = <BrowserDropdown fullScreenFunc={ this.fullScreen.bind(this) }
aboutFunc={ this.showAbout.bind(this) }
settingsFunc={ this.showSettings.bind(this) }
logoutFunc={ this.logout.bind(this) } />
} else {
loginButton = <a className='btn btn-danger' href='/minio/login'>Login</a>
}
if (web.LoggedIn()) {
if (!(used === 0 && free === 0)) {
storageUsageDetails = <div className="feh-usage">
<div className="fehu-chart">
<div style={ { width: usedPercent } }></div>
</div>
<ul>
<li>
<span>Used: </span>
{ humanize.filesize(total - free) }
</li>
<li className="pull-right">
<span>Free: </span>
{ humanize.filesize(total - used) }
</li>
</ul>
</div>
}
}
let createButton = ''
if (web.LoggedIn()) {
createButton = <Dropdown dropup className="feb-actions" id="fe-action-toggle">
<Dropdown.Toggle noCaret className="feba-toggle">
<span><i className="fa fa-plus"></i></span>
</Dropdown.Toggle>
<Dropdown.Menu>
<OverlayTrigger placement="left" overlay={ uploadTooltip }>
<a href="#" className="feba-btn feba-upload">
<input type="file"
onChange={ this.uploadFile.bind(this) }
style={ { display: 'none' } }
id="file-input"></input>
<label htmlFor="file-input"> <i className="fa fa-cloud-upload"></i> </label>
</a>
</OverlayTrigger>
<OverlayTrigger placement="left" overlay={ makeBucketTooltip }>
<a href="#" className="feba-btn feba-bucket" onClick={ this.showMakeBucketModal.bind(this) }><i className="fa fa-hdd-o"></i></a>
</OverlayTrigger>
</Dropdown.Menu>
</Dropdown>
} else {
if (prefixWritable)
createButton = <Dropdown dropup className="feb-actions" id="fe-action-toggle">
<Dropdown.Toggle noCaret className="feba-toggle">
<span><i className="fa fa-plus"></i></span>
</Dropdown.Toggle>
<Dropdown.Menu>
<OverlayTrigger placement="left" overlay={ uploadTooltip }>
<a href="#" className="feba-btn feba-upload">
<input type="file"
onChange={ this.uploadFile.bind(this) }
style={ { display: 'none' } }
id="file-input"></input>
<label htmlFor="file-input"> <i className="fa fa-cloud-upload"></i> </label>
</a>
</OverlayTrigger>
</Dropdown.Menu>
</Dropdown>
}
return (
<div className={ classNames({
'file-explorer': true,
'toggled': sidebarStatus
}) }>
<SideBar searchBuckets={ this.searchBuckets.bind(this) }
selectBucket={ this.selectBucket.bind(this) }
clickOutside={ this.hideSidebar.bind(this) }
showPolicy={ this.showBucketPolicy.bind(this) } />
<div className="fe-body">
<div className={ 'list-actions' + (classNames({
' list-actions-toggled': checkedObjects.length > 0
})) }>
<span className="la-label"><i className="fa fa-check-circle" /> { checkedObjects.length } Objects selected</span>
<span className="la-actions pull-right"><button onClick={ this.downloadSelected.bind(this) }> Download all as zip </button></span>
<span className="la-actions pull-right"><button onClick={ this.showDeleteConfirmation.bind(this) }> Delete selected </button></span>
<i className="la-close fa fa-times" onClick={ this.clearSelected.bind(this) }></i>
</div>
<Dropzone>
{ alertBox }
<header className="fe-header-mobile hidden-lg hidden-md">
<div id="feh-trigger" className={ 'feh-trigger ' + (classNames({
'feht-toggled': sidebarStatus
})) } onClick={ this.toggleSidebar.bind(this, !sidebarStatus) }>
<div className="feht-lines">
<div className="top"></div>
<div className="center"></div>
<div className="bottom"></div>
</div>
</div>
<img className="mh-logo" src={ logo } alt="" />
</header>
<header className="fe-header">
<Path selectPrefix={ this.selectPrefix.bind(this) } />
{ storageUsageDetails }
<ul className="feh-actions">
<BrowserUpdate />
{ loginButton }
{ browserDropdownButton }
</ul>
</header>
<div className="feb-container">
<header className="fesl-row" data-type="folder">
<div className="fesl-item fesl-item-icon"></div>
<div className="fesl-item fesl-item-name" onClick={ this.sortObjectsByName.bind(this) } data-sort="name">
Name
<i className={ classNames({
'fesli-sort': true,
'fa': true,
'fa-sort-alpha-desc': sortNameOrder,
'fa-sort-alpha-asc': !sortNameOrder
}) } />
</div>
<div className="fesl-item fesl-item-size" onClick={ this.sortObjectsBySize.bind(this) } data-sort="size">
Size
<i className={ classNames({
'fesli-sort': true,
'fa': true,
'fa-sort-amount-desc': sortSizeOrder,
'fa-sort-amount-asc': !sortSizeOrder
}) } />
</div>
<div className="fesl-item fesl-item-modified" onClick={ this.sortObjectsByDate.bind(this) } data-sort="last-modified">
Last Modified
<i className={ classNames({
'fesli-sort': true,
'fa': true,
'fa-sort-numeric-desc': sortDateOrder,
'fa-sort-numeric-asc': !sortDateOrder
}) } />
</div>
<div className="fesl-item fesl-item-actions"></div>
</header>
</div>
<div className="feb-container">
<InfiniteScroll loadMore={ this.listObjects.bind(this) }
hasMore={ istruncated }
useWindow={ true }
initialLoad={ false }>
<ObjectsList dataType={ this.dataType.bind(this) }
selectPrefix={ this.selectPrefix.bind(this) }
showDeleteConfirmation={ this.showDeleteConfirmation.bind(this) }
shareObject={ this.shareObject.bind(this) }
checkObject={ this.checkObject.bind(this) }
checkedObjectsArray={ checkedObjects } />
</InfiniteScroll>
<div className="text-center" style={ { display: (istruncated && currentBucket) ? 'block' : 'none' } }>
<span>Loading...</span>
</div>
</div>
<UploadModal />
{ createButton }
<Modal className="modal-create-bucket"
bsSize="small"
animation={ false }
show={ showMakeBucketModal }
onHide={ this.hideMakeBucketModal.bind(this) }>
<button className="close close-alt" onClick={ this.hideMakeBucketModal.bind(this) }>
<span>×</span>
</button>
<ModalBody>
<form onSubmit={ this.makeBucket.bind(this) }>
<div className="input-group">
<input className="ig-text"
type="text"
ref="makeBucketRef"
placeholder="Bucket Name"
autoFocus/>
<i className="ig-helpers"></i>
</div>
</form>
</ModalBody>
</Modal>
<Modal className="modal-about modal-dark"
animation={ false }
show={ showAbout }
onHide={ this.hideAbout.bind(this) }>
<button className="close" onClick={ this.hideAbout.bind(this) }>
<span>×</span>
</button>
<div className="ma-inner">
<div className="mai-item hidden-xs">
<a href="https://minio.io" target="_blank"><img className="maii-logo" src={ logo } alt="" /></a>
</div>
<div className="mai-item">
<ul className="maii-list">
<li>
<div>
Version
</div>
<small>{ version }</small>
</li>
<li>
<div>
Memory
</div>
<small>{ memory }</small>
</li>
<li>
<div>
Platform
</div>
<small>{ platform }</small>
</li>
<li>
<div>
Runtime
</div>
<small>{ runtime }</small>
</li>
</ul>
</div>
</div>
</Modal>
<Modal className="modal-policy"
animation={ false }
show={ showBucketPolicy }
onHide={ this.hideBucketPolicy.bind(this) }>
<ModalHeader>
Bucket Policy (
{ currentBucket })
<button className="close close-alt" onClick={ this.hideBucketPolicy.bind(this) }>
<span>×</span>
</button>
</ModalHeader>
<div className="pm-body">
<PolicyInput bucket={ currentBucket } />
{ policies.map((policy, i) => <Policy key={ i } prefix={ policy.prefix } policy={ policy.policy } />
) }
</div>
</Modal>
<ConfirmModal show={ deleteConfirmation.show }
icon='fa fa-exclamation-triangle mci-red'
text='Are you sure you want to delete?'
sub='This cannot be undone!'
okText='Delete'
cancelText='Cancel'
okHandler={ this.removeObject.bind(this) }
cancelHandler={ this.hideDeleteConfirmation.bind(this) }>
</ConfirmModal>
<Modal show={ shareObject.show }
animation={ false }
onHide={ this.hideShareObjectModal.bind(this) }
bsSize="small">
<ModalHeader>
Share Object
</ModalHeader>
<ModalBody>
<div className="input-group copy-text">
<label>
Shareable Link
</label>
<input type="text"
ref="copyTextInput"
readOnly="readOnly"
value={ window.location.protocol + '//' + shareObject.url }
onClick={ this.selectTexts.bind(this) } />
</div>
<div className="input-group" style={ { display: web.LoggedIn() ? 'block' : 'none' } }>
<label>
Expires in (Max 7 days)
</label>
<div className="set-expire">
<div className="set-expire-item">
<i className="set-expire-increase" onClick={ this.handleExpireValue.bind(this, 'expireDays', 1, shareObject.object) } />
<div className="set-expire-title">
Days
</div>
<div className="set-expire-value">
<input ref="expireDays"
type="number"
min={ 0 }
max={ 7 }
defaultValue={ 5 }
readOnly="readOnly"
/>
</div>
<i className="set-expire-decrease" onClick={ this.handleExpireValue.bind(this, 'expireDays', -1, shareObject.object) } />
</div>
<div className="set-expire-item">
<i className="set-expire-increase" onClick={ this.handleExpireValue.bind(this, 'expireHours', 1, shareObject.object) } />
<div className="set-expire-title">
Hours
</div>
<div className="set-expire-value">
<input ref="expireHours"
type="number"
min={ 0 }
max={ 23 }
defaultValue={ 0 }
readOnly="readOnly"
/>
</div>
<i className="set-expire-decrease" onClick={ this.handleExpireValue.bind(this, 'expireHours', -1, shareObject.object) } />
</div>
<div className="set-expire-item">
<i className="set-expire-increase" onClick={ this.handleExpireValue.bind(this, 'expireMins', 1, shareObject.object) } />
<div className="set-expire-title">
Minutes
</div>
<div className="set-expire-value">
<input ref="expireMins"
type="number"
min={ 0 }
max={ 59 }
defaultValue={ 0 }
readOnly="readOnly"
/>
</div>
<i className="set-expire-decrease" onClick={ this.handleExpireValue.bind(this, 'expireMins', -1, shareObject.object) } />
</div>
</div>
</div>
</ModalBody>
<div className="modal-footer">
<CopyToClipboard text={ window.location.protocol + '//' + shareObject.url } onCopy={ this.showMessage.bind(this) }>
<button className="btn btn-success">
Copy Link
</button>
</CopyToClipboard>
<button className="btn btn-link" onClick={ this.hideShareObjectModal.bind(this) }>
Cancel
</button>
</div>
</Modal>
{ settingsModal }
</Dropzone>
</div>
</div>
)
}
}

View File

@@ -0,0 +1,56 @@
/*
* Minio Cloud Storage (C) 2016, 2017 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import connect from 'react-redux/lib/components/connect'
import Dropdown from 'react-bootstrap/lib/Dropdown'
let BrowserDropdown = ({fullScreenFunc, aboutFunc, settingsFunc, logoutFunc}) => {
return (
<li>
<Dropdown pullRight id="top-right-menu">
<Dropdown.Toggle noCaret>
<i className="fa fa-reorder"></i>
</Dropdown.Toggle>
<Dropdown.Menu className="dropdown-menu-right">
<li>
<a target="_blank" href="https://github.com/minio/minio">Github <i className="fa fa-github"></i></a>
</li>
<li>
<a href="" onClick={ fullScreenFunc }>Fullscreen <i className="fa fa-expand"></i></a>
</li>
<li>
<a target="_blank" href="https://docs.minio.io/">Documentation <i className="fa fa-book"></i></a>
</li>
<li>
<a target="_blank" href="https://slack.minio.io">Ask for help <i className="fa fa-question-circle"></i></a>
</li>
<li>
<a href="" onClick={ aboutFunc }>About <i className="fa fa-info-circle"></i></a>
</li>
<li>
<a href="" onClick={ settingsFunc }>Settings <i className="fa fa-cog"></i></a>
</li>
<li>
<a href="" onClick={ logoutFunc }>Sign Out <i className="fa fa-sign-out"></i></a>
</li>
</Dropdown.Menu>
</Dropdown>
</li>
)
}
export default connect(state => state)(BrowserDropdown)

View File

@@ -0,0 +1,42 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import connect from 'react-redux/lib/components/connect'
import Tooltip from 'react-bootstrap/lib/Tooltip'
import OverlayTrigger from 'react-bootstrap/lib/OverlayTrigger'
let BrowserUpdate = ({latestUiVersion}) => {
// Don't show an update if we're already updated!
if (latestUiVersion === currentUiVersion) return ( <noscript></noscript> )
return (
<li className="hidden-xs hidden-sm">
<a href="">
<OverlayTrigger placement="left" overlay={ <Tooltip id="tt-version-update">
New update available. Click to refresh.
</Tooltip> }> <i className="fa fa-refresh"></i> </OverlayTrigger>
</a>
</li>
)
}
export default connect(state => {
return {
latestUiVersion: state.latestUiVersion
}
})(BrowserUpdate)

View File

@@ -0,0 +1,50 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import Modal from 'react-bootstrap/lib/Modal'
import ModalBody from 'react-bootstrap/lib/ModalBody'
let ConfirmModal = ({baseClass, icon, text, sub, okText, cancelText, okHandler, cancelHandler, show}) => {
return (
<Modal bsSize="small"
animation={ false }
show={ show }
className={ "modal-confirm " + (baseClass || '') }>
<ModalBody>
<div className="mc-icon">
<i className={ icon }></i>
</div>
<div className="mc-text">
{ text }
</div>
<div className="mc-sub">
{ sub }
</div>
</ModalBody>
<div className="modal-footer">
<button className="btn btn-danger" onClick={ okHandler }>
{ okText }
</button>
<button className="btn btn-link" onClick={ cancelHandler }>
{ cancelText }
</button>
</div>
</Modal>
)
}
export default ConfirmModal

View File

@@ -0,0 +1,66 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import ReactDropzone from 'react-dropzone'
import * as actions from '../actions'
// Dropzone is a drag-and-drop element for uploading files. It will create a
// landing zone of sorts that automatically receives the files.
export default class Dropzone extends React.Component {
onDrop(files) {
// FIXME: Currently you can upload multiple files, but only one abort
// modal will be shown, and progress updates will only occur for one
// file at a time. See #171.
files.forEach(file => {
let req = new XMLHttpRequest()
// Dispatch the upload.
web.dispatch(actions.uploadFile(file, req))
})
}
render() {
// Overwrite the default styling from react-dropzone; otherwise it
// won't handle child elements correctly.
const style = {
height: '100%',
borderWidth: '0',
borderStyle: 'dashed',
borderColor: '#fff'
}
const activeStyle = {
borderWidth: '2px',
borderColor: '#777'
}
const rejectStyle = {
backgroundColor: '#ffdddd'
}
// disableClick means that it won't trigger a file upload box when
// the user clicks on a file.
return (
<ReactDropzone style={ style }
activeStyle={ activeStyle }
rejectStyle={ rejectStyle }
disableClick={ true }
onDrop={ this.onDrop }>
{ this.props.children }
</ReactDropzone>
)
}
}

View File

@@ -0,0 +1,49 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
let InputGroup = ({label, id, name, value, onChange, type, spellCheck, required, readonly, autoComplete, align, className}) => {
var input = <input id={ id }
name={ name }
value={ value }
onChange={ onChange }
className="ig-text"
type={ type }
spellCheck={ spellCheck }
required={ required }
autoComplete={ autoComplete } />
if (readonly)
input = <input id={ id }
name={ name }
value={ value }
onChange={ onChange }
className="ig-text"
type={ type }
spellCheck={ spellCheck }
required={ required }
autoComplete={ autoComplete }
disabled />
return <div className={ "input-group " + align + ' ' + className }>
{ input }
<i className="ig-helpers"></i>
<label className="ig-label">
{ label }
</label>
</div>
}
export default InputGroup

View File

@@ -0,0 +1,128 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import classNames from 'classnames'
import logo from '../../img/logo.svg'
import Alert from 'react-bootstrap/lib/Alert'
import * as actions from '../actions'
import InputGroup from '../components/InputGroup'
export default class Login extends React.Component {
handleSubmit(event) {
event.preventDefault()
const {web, dispatch, loginRedirectPath} = this.props
let message = ''
if (!document.getElementById('accessKey').value) {
message = 'Secret Key cannot be empty'
}
if (!document.getElementById('secretKey').value) {
message = 'Access Key cannot be empty'
}
if (message) {
dispatch(actions.showAlert({
type: 'danger',
message
}))
return
}
web.Login({
username: document.getElementById('accessKey').value,
password: document.getElementById('secretKey').value
})
.then((res) => {
this.context.router.push(loginRedirectPath)
})
.catch(e => {
dispatch(actions.setLoginError())
dispatch(actions.showAlert({
type: 'danger',
message: e.message
}))
})
}
componentWillMount() {
const {dispatch} = this.props
// Clear out any stale message in the alert of previous page
dispatch(actions.showAlert({
type: 'danger',
message: ''
}))
document.body.classList.add('is-guest')
}
componentWillUnmount() {
document.body.classList.remove('is-guest')
}
hideAlert() {
const {dispatch} = this.props
dispatch(actions.hideAlert())
}
render() {
const {alert} = this.props
let alertBox = <Alert className={ 'alert animated ' + (alert.show ? 'fadeInDown' : 'fadeOutUp') } bsStyle={ alert.type } onDismiss={ this.hideAlert.bind(this) }>
<div className='text-center'>
{ alert.message }
</div>
</Alert>
// Make sure you don't show a fading out alert box on the initial web-page load.
if (!alert.message)
alertBox = ''
return (
<div className="login">
{ alertBox }
<div className="l-wrap">
<form onSubmit={ this.handleSubmit.bind(this) }>
<InputGroup className="ig-dark"
label="Access Key"
id="accessKey"
name="username"
type="text"
spellCheck="false"
required="required"
autoComplete="username">
</InputGroup>
<InputGroup className="ig-dark"
label="Secret Key"
id="secretKey"
name="password"
type="password"
spellCheck="false"
required="required"
autoComplete="new-password">
</InputGroup>
<button className="lw-btn" type="submit">
<i className="fa fa-sign-in"></i>
</button>
</form>
</div>
<div className="l-footer">
<a className="lf-logo" href=""><img src={ logo } alt="" /></a>
<div className="lf-server">
{ window.location.host }
</div>
</div>
</div>
)
}
}
Login.contextTypes = {
router: React.PropTypes.object.isRequired
}

View File

@@ -0,0 +1,96 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import Moment from 'moment'
import humanize from 'humanize'
import connect from 'react-redux/lib/components/connect'
import Dropdown from 'react-bootstrap/lib/Dropdown'
let ObjectsList = ({objects, currentPath, selectPrefix, dataType, showDeleteConfirmation, shareObject, loadPath, checkObject, checkedObjectsArray}) => {
const list = objects.map((object, i) => {
let size = object.name.endsWith('/') ? '-' : humanize.filesize(object.size)
let lastModified = object.name.endsWith('/') ? '-' : Moment(object.lastModified).format('lll')
let loadingClass = loadPath === `${currentPath}${object.name}` ? 'fesl-loading' : ''
let actionButtons = ''
let deleteButton = ''
if (web.LoggedIn())
deleteButton = <a href="" className="fiad-action" onClick={ (e) => showDeleteConfirmation(e, `${currentPath}${object.name}`) }><i className="fa fa-trash"></i></a>
if (!checkedObjectsArray.length > 0) {
if (!object.name.endsWith('/')) {
actionButtons = <Dropdown id={ "fia-dropdown-" + object.name.replace('.', '-') }>
<Dropdown.Toggle noCaret className="fia-toggle"></Dropdown.Toggle>
<Dropdown.Menu>
<a href="" className="fiad-action" onClick={ (e) => shareObject(e, `${currentPath}${object.name}`) }><i className="fa fa-copy"></i></a>
{ deleteButton }
</Dropdown.Menu>
</Dropdown>
}
}
let activeClass = ''
let isChecked = ''
if (checkedObjectsArray.indexOf(object.name) > -1) {
activeClass = ' fesl-row-selected'
isChecked = true
}
return (
<div key={ i } className={ "fesl-row " + loadingClass + activeClass } data-type={ dataType(object.name, object.contentType) }>
<div className="fesl-item fesl-item-icon">
<div className="fi-select">
<input type="checkbox"
name={ object.name }
checked={ isChecked }
onChange={ (e) => checkObject(e, object.name) } />
<i className="fis-icon"></i>
<i className="fis-helper"></i>
</div>
</div>
<div className="fesl-item fesl-item-name">
<a href="" onClick={ (e) => selectPrefix(e, `${currentPath}${object.name}`) }>
{ object.name }
</a>
</div>
<div className="fesl-item fesl-item-size">
{ size }
</div>
<div className="fesl-item fesl-item-modified">
{ lastModified }
</div>
<div className="fesl-item fesl-item-actions">
{ actionButtons }
</div>
</div>
)
})
return (
<div>
{ list }
</div>
)
}
// Subscribe it to state changes.
export default connect(state => {
return {
objects: state.objects,
currentPath: state.currentPath,
loadPath: state.loadPath
}
})(ObjectsList)

View File

@@ -0,0 +1,41 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import connect from 'react-redux/lib/components/connect'
let Path = ({currentBucket, currentPath, selectPrefix}) => {
let dirPath = []
let path = ''
if (currentPath) {
path = currentPath.split('/').map((dir, i) => {
dirPath.push(dir)
let dirPath_ = dirPath.join('/') + '/'
return <span key={ i }><a href="" onClick={ (e) => selectPrefix(e, dirPath_) }>{ dir }</a></span>
})
}
return (
<h2><span className="main"><a onClick={ (e) => selectPrefix(e, '') } href="">{ currentBucket }</a></span>{ path }</h2>
)
}
export default connect(state => {
return {
currentBucket: state.currentBucket,
currentPath: state.currentPath
}
})(Path)

View File

@@ -0,0 +1,80 @@
import { READ_ONLY, WRITE_ONLY, READ_WRITE } from '../constants'
import React, { Component, PropTypes } from 'react'
import connect from 'react-redux/lib/components/connect'
import classnames from 'classnames'
import * as actions from '../actions'
class Policy extends Component {
constructor(props, context) {
super(props, context)
this.state = {}
}
handlePolicyChange(e) {
this.setState({
policy: {
policy: e.target.value
}
})
}
removePolicy(e) {
e.preventDefault()
const {dispatch, currentBucket, prefix} = this.props
let newPrefix = prefix.replace(currentBucket + '/', '')
newPrefix = newPrefix.replace('*', '')
web.SetBucketPolicy({
bucketName: currentBucket,
prefix: newPrefix,
policy: 'none'
})
.then(() => {
dispatch(actions.setPolicies(this.props.policies.filter(policy => policy.prefix != prefix)))
})
.catch(e => dispatch(actions.showAlert({
type: 'danger',
message: e.message,
})))
}
render() {
const {policy, prefix, currentBucket} = this.props
let newPrefix = prefix.replace(currentBucket + '/', '')
newPrefix = newPrefix.replace('*', '')
if (!newPrefix)
newPrefix = '*'
return (
<div className="pmb-list">
<div className="pmbl-item">
{ newPrefix }
</div>
<div className="pmbl-item">
<select className="form-control"
disabled
value={ policy }
onChange={ this.handlePolicyChange.bind(this) }>
<option value={ READ_ONLY }>
Read Only
</option>
<option value={ WRITE_ONLY }>
Write Only
</option>
<option value={ READ_WRITE }>
Read and Write
</option>
</select>
</div>
<div className="pmbl-item">
<button className="btn btn-block btn-danger" onClick={ this.removePolicy.bind(this) }>
Remove
</button>
</div>
</div>
)
}
}
export default connect(state => state)(Policy)

View File

@@ -0,0 +1,98 @@
import { READ_ONLY, WRITE_ONLY, READ_WRITE } from '../constants'
import React, { Component, PropTypes } from 'react'
import connect from 'react-redux/lib/components/connect'
import classnames from 'classnames'
import * as actions from '../actions'
class PolicyInput extends Component {
componentDidMount() {
const {web, dispatch} = this.props
this.prefix.focus()
web.ListAllBucketPolicies({
bucketName: this.props.currentBucket
}).then(res => {
let policies = res.policies
if (policies) dispatch(actions.setPolicies(policies))
}).catch(err => {
dispatch(actions.showAlert({
type: 'danger',
message: err.message
}))
})
}
componentWillUnmount() {
const {dispatch} = this.props
dispatch(actions.setPolicies([]))
}
handlePolicySubmit(e) {
e.preventDefault()
const {web, dispatch, currentBucket} = this.props
let prefix = currentBucket + '/' + this.prefix.value
let policy = this.policy.value
if (!prefix.endsWith('*')) prefix = prefix + '*'
let prefixAlreadyExists = this.props.policies.some(elem => prefix === elem.prefix)
if (prefixAlreadyExists) {
dispatch(actions.showAlert({
type: 'danger',
message: "Policy for this prefix already exists."
}))
return
}
web.SetBucketPolicy({
bucketName: this.props.currentBucket,
prefix: this.prefix.value,
policy: this.policy.value
})
.then(() => {
dispatch(actions.setPolicies([{
policy, prefix
}, ...this.props.policies]))
this.prefix.value = ''
})
.catch(e => dispatch(actions.showAlert({
type: 'danger',
message: e.message,
})))
}
render() {
return (
<header className="pmb-list">
<div className="pmbl-item">
<input type="text"
ref={ prefix => this.prefix = prefix }
className="form-control"
placeholder="Prefix"
editable={ true } />
</div>
<div className="pmbl-item">
<select ref={ policy => this.policy = policy } className="form-control">
<option value={ READ_ONLY }>
Read Only
</option>
<option value={ WRITE_ONLY }>
Write Only
</option>
<option value={ READ_WRITE }>
Read and Write
</option>
</select>
</div>
<div className="pmbl-item">
<button className="btn btn-block btn-primary" onClick={ this.handlePolicySubmit.bind(this) }>
Add
</button>
</div>
</header>
)
}
}
export default connect(state => state)(PolicyInput)

View File

@@ -0,0 +1,204 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import connect from 'react-redux/lib/components/connect'
import * as actions from '../actions'
import Tooltip from 'react-bootstrap/lib/Tooltip'
import Modal from 'react-bootstrap/lib/Modal'
import ModalBody from 'react-bootstrap/lib/ModalBody'
import ModalHeader from 'react-bootstrap/lib/ModalHeader'
import OverlayTrigger from 'react-bootstrap/lib/OverlayTrigger'
import InputGroup from './InputGroup'
class SettingsModal extends React.Component {
// When the settings are shown, it loads the access key and secret key.
componentWillMount() {
const {web, dispatch} = this.props
const {serverInfo} = this.props
let accessKeyEnv = ''
let secretKeyEnv = ''
// Check environment variables first.
if (serverInfo.info.isEnvCreds) {
dispatch(actions.setSettings({
accessKey: 'xxxxxxxxx',
secretKey: 'xxxxxxxxx',
keysReadOnly: true
}))
} else {
web.GetAuth()
.then(data => {
dispatch(actions.setSettings({
accessKey: data.accessKey,
secretKey: data.secretKey
}))
})
}
}
// When they are re-hidden, the keys are unloaded from memory.
componentWillUnmount() {
const {dispatch} = this.props
dispatch(actions.setSettings({
accessKey: '',
secretKey: '',
secretKeyVisible: false
}))
dispatch(actions.hideSettings())
}
// Handle field changes from inside the modal.
accessKeyChange(e) {
const {dispatch} = this.props
dispatch(actions.setSettings({
accessKey: e.target.value
}))
}
secretKeyChange(e) {
const {dispatch} = this.props
dispatch(actions.setSettings({
secretKey: e.target.value
}))
}
secretKeyVisible(secretKeyVisible) {
const {dispatch} = this.props
dispatch(actions.setSettings({
secretKeyVisible
}))
}
// Save the auth params and set them.
setAuth(e) {
e.preventDefault()
const {web, dispatch} = this.props
let accessKey = document.getElementById('accessKey').value
let secretKey = document.getElementById('secretKey').value
web.SetAuth({
accessKey,
secretKey
})
.then(data => {
dispatch(actions.setSettings({
accessKey: '',
secretKey: '',
secretKeyVisible: false
}))
dispatch(actions.hideSettings())
dispatch(actions.showAlert({
type: 'success',
message: 'Changed credentials'
}))
})
.catch(err => {
dispatch(actions.setSettings({
accessKey: '',
secretKey: '',
secretKeyVisible: false
}))
dispatch(actions.hideSettings())
dispatch(actions.showAlert({
type: 'danger',
message: err.message
}))
})
}
generateAuth(e) {
e.preventDefault()
const {dispatch} = this.props
web.GenerateAuth()
.then(data => {
dispatch(actions.setSettings({
secretKeyVisible: true
}))
dispatch(actions.setSettings({
accessKey: data.accessKey,
secretKey: data.secretKey
}))
})
}
hideSettings(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.hideSettings())
}
render() {
let {settings} = this.props
return (
<Modal bsSize="sm" animation={ false } show={ true }>
<ModalHeader>
Change Password
</ModalHeader>
<ModalBody className="m-t-20">
<InputGroup value={ settings.accessKey }
onChange={ this.accessKeyChange.bind(this) }
id="accessKey"
label="Access Key"
name="accesskey"
type="text"
spellCheck="false"
required="required"
autoComplete="false"
align="ig-left"
readonly={ settings.keysReadOnly }></InputGroup>
<i onClick={ this.secretKeyVisible.bind(this, !settings.secretKeyVisible) } className={ "toggle-password fa fa-eye " + (settings.secretKeyVisible ? "toggled" : "") } />
<InputGroup value={ settings.secretKey }
onChange={ this.secretKeyChange.bind(this) }
id="secretKey"
label="Secret Key"
name="accesskey"
type={ settings.secretKeyVisible ? "text" : "password" }
spellCheck="false"
required="required"
autoComplete="false"
align="ig-left"
readonly={ settings.keysReadOnly }></InputGroup>
</ModalBody>
<div className="modal-footer">
<button className={ "btn btn-primary " + (settings.keysReadOnly ? "hidden" : "") } onClick={ this.generateAuth.bind(this) }>
Generate
</button>
<button href="" className={ "btn btn-success " + (settings.keysReadOnly ? "hidden" : "") } onClick={ this.setAuth.bind(this) }>
Update
</button>
<button href="" className="btn btn-link" onClick={ this.hideSettings.bind(this) }>
Cancel
</button>
</div>
</Modal>
)
}
}
export default connect(state => {
return {
web: state.web,
settings: state.settings,
serverInfo: state.serverInfo
}
})(SettingsModal)

View File

@@ -0,0 +1,85 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import classNames from 'classnames'
import ClickOutHandler from 'react-onclickout'
import Scrollbars from 'react-custom-scrollbars/lib/Scrollbars'
import connect from 'react-redux/lib/components/connect'
import logo from '../../img/logo.svg'
let SideBar = ({visibleBuckets, loadBucket, currentBucket, selectBucket, searchBuckets, sidebarStatus, clickOutside, showPolicy}) => {
const list = visibleBuckets.map((bucket, i) => {
return <li className={ classNames({
'active': bucket === currentBucket
}) } key={ i } onClick={ (e) => selectBucket(e, bucket) }>
<a href="" className={ classNames({
'fesli-loading': bucket === loadBucket
}) }>
{ bucket }
</a>
<i className="fesli-trigger" onClick={ showPolicy }></i>
</li>
})
return (
<ClickOutHandler onClickOut={ clickOutside }>
<div className={ classNames({
'fe-sidebar': true,
'toggled': sidebarStatus
}) }>
<div className="fes-header clearfix hidden-sm hidden-xs">
<img src={ logo } alt="" />
<h2>Minio Browser</h2>
</div>
<div className="fes-list">
<div className="input-group ig-dark ig-left ig-search" style={ { display: web.LoggedIn() ? 'block' : 'none' } }>
<input className="ig-text"
type="text"
onChange={ searchBuckets }
placeholder="Search Buckets..." />
<i className="ig-helpers"></i>
</div>
<div className="fesl-inner">
<Scrollbars renderScrollbarVertical={ props => <div className="scrollbar-vertical" /> }>
<ul>
{ list }
</ul>
</Scrollbars>
</div>
</div>
<div className="fes-host">
<i className="fa fa-globe"></i>
<a href="/">
{ window.location.host }
</a>
</div>
</div>
</ClickOutHandler>
)
}
// Subscribe it to state changes that affect only the sidebar.
export default connect(state => {
return {
visibleBuckets: state.visibleBuckets,
loadBucket: state.loadBucket,
currentBucket: state.currentBucket,
sidebarStatus: state.sidebarStatus
}
})(SideBar)

View File

@@ -0,0 +1,141 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import React from 'react'
import humanize from 'humanize'
import classNames from 'classnames'
import connect from 'react-redux/lib/components/connect'
import ProgressBar from 'react-bootstrap/lib/ProgressBar'
import ConfirmModal from './ConfirmModal'
import * as actions from '../actions'
// UploadModal is a modal that handles multiple file uploads.
// During the upload, it displays a progress bar, and can transform into an
// abort modal if the user decides to abort the uploads.
class UploadModal extends React.Component {
// Abort all the current uploads.
abortUploads(e) {
e.preventDefault()
const {dispatch, uploads} = this.props
for (var slug in uploads) {
let upload = uploads[slug]
upload.xhr.abort()
dispatch(actions.stopUpload({
slug
}))
}
this.hideAbort(e)
}
// Show the abort modal instead of the progress modal.
showAbort(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.setShowAbortModal(true))
}
// Show the progress modal instead of the abort modal.
hideAbort(e) {
e.preventDefault()
const {dispatch} = this.props
dispatch(actions.setShowAbortModal(false))
}
render() {
const {uploads, showAbortModal} = this.props
// Show the abort modal.
if (showAbortModal) {
let baseClass = classNames({
'abort-upload': true
})
let okIcon = classNames({
'fa': true,
'fa-times': true
})
let cancelIcon = classNames({
'fa': true,
'fa-cloud-upload': true
})
return (
<ConfirmModal show={ true }
baseClass={ baseClass }
text='Abort uploads in progress?'
icon='fa fa-info-circle mci-amber'
sub='This cannot be undone!'
okText='Abort'
okIcon={ okIcon }
cancelText='Upload'
cancelIcon={ cancelIcon }
okHandler={ this.abortUploads.bind(this) }
cancelHandler={ this.hideAbort.bind(this) }>
</ConfirmModal>
)
}
// If we don't have any files uploading, don't show anything.
let numberUploading = Object.keys(uploads).length
if (numberUploading == 0)
return ( <noscript></noscript> )
let totalLoaded = 0
let totalSize = 0
// Iterate over each upload, adding together the total size and that
// which has been uploaded.
for (var slug in uploads) {
let upload = uploads[slug]
totalLoaded += upload.loaded
totalSize += upload.size
}
let percent = (totalLoaded / totalSize) * 100
// If more than one: "Uploading files (5)..."
// If only one: "Uploading myfile.txt..."
let text = 'Uploading ' + (numberUploading == 1 ? `'${uploads[Object.keys(uploads)[0]].name}'` : `files (${numberUploading})`) + '...'
return (
<div className="alert alert-info progress animated fadeInUp ">
<button type="button" className="close" onClick={ this.showAbort.bind(this) }>
<span>×</span>
</button>
<div className="text-center">
<small>{ text }</small>
</div>
<ProgressBar now={ percent } />
<div className="text-center">
<small>{ humanize.filesize(totalLoaded) } ({ percent.toFixed(2) } %)</small>
</div>
</div>
)
}
}
export default connect(state => {
return {
uploads: state.uploads,
showAbortModal: state.showAbortModal
}
})(UploadModal)

View File

@@ -0,0 +1,54 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
import React from 'react'
import ReactTestUtils, {renderIntoDocument} from 'react-addons-test-utils'
import expect from 'expect'
import Login from '../Login'
describe('Login', () => {
it('it should have empty credentials', () => {
const alert = {
show: false
}
const dispatch = () => {}
let loginComponent = renderIntoDocument(<Login alert={alert} dispatch={dispatch} />)
const accessKey = document.getElementById('accessKey')
const secretKey = document.getElementById('secretKey')
// Validate default value.
expect(accessKey.value).toEqual('')
expect(secretKey.value).toEqual('')
})
it('it should set accessKey and secretKey', () => {
const alert = {
show: false
}
const dispatch = () => {}
let loginComponent = renderIntoDocument(<Login alert={alert} dispatch={dispatch} />)
let accessKey = loginComponent.refs.accessKey
let secretKey = loginComponent.refs.secretKey
accessKey.value = 'demo-username'
secretKey.value = 'demo-password'
ReactTestUtils.Simulate.change(accessKey)
ReactTestUtils.Simulate.change(secretKey)
// Validate if the change has occurred.
expect(loginComponent.refs.accessKey.value).toEqual('demo-username')
expect(loginComponent.refs.secretKey.value).toEqual('demo-password')
})
});
*/

View File

@@ -0,0 +1,23 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// File for all the browser constants.
// minioBrowserPrefix absolute path.
export const minioBrowserPrefix = '/minio'
export const READ_ONLY = 'readonly'
export const WRITE_ONLY = 'writeonly'
export const READ_WRITE = 'readwrite'

91
browser/app/js/jsonrpc.js Normal file
View File

@@ -0,0 +1,91 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import SuperAgent from 'superagent-es6-promise';
import url from 'url'
import Moment from 'moment'
export default class JSONrpc {
constructor(params) {
this.endpoint = params.endpoint
this.namespace = params.namespace
this.version = '2.0';
const parsedUrl = url.parse(this.endpoint)
this.host = parsedUrl.hostname
this.path = parsedUrl.path
this.port = parsedUrl.port
switch (parsedUrl.protocol) {
case 'http:': {
this.scheme = 'http'
if (parsedUrl.port === 0) {
this.port = 80
}
break
}
case 'https:': {
this.scheme = 'https'
if (parsedUrl.port === 0) {
this.port = 443
}
break
}
default: {
throw new Error('Unknown protocol: ' + parsedUrl.protocol)
}
}
}
// call('Get', {id: NN, params: [...]}, function() {})
call(method, options, token) {
if (!options) {
options = {}
}
if (!options.id) {
options.id = 1;
}
if (!options.params) {
options.params = {};
}
const dataObj = {
id: options.id,
jsonrpc: this.version,
params: options.params ? options.params : {},
method: this.namespace ? this.namespace + '.' + method : method
}
let requestParams = {
host: this.host,
port: this.port,
path: this.path,
scheme: this.scheme,
method: 'POST',
headers: {
'Content-Type': 'application/json',
'x-amz-date': Moment().utc().format('YYYYMMDDTHHmmss') + 'Z'
}
}
if (token) {
requestParams.headers.Authorization = 'Bearer ' + token
}
let req = SuperAgent.post(this.endpoint)
for (let key in requestParams.headers) {
req.set(key, requestParams.headers[key])
}
// req.set('Access-Control-Allow-Origin', 'http://localhost:8080')
return req.send(JSON.stringify(dataObj)).then(res => res)
}
}

106
browser/app/js/mime.js Normal file
View File

@@ -0,0 +1,106 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import mimedb from 'mime-types'
const isFolder = (name, contentType) => {
if (name.endsWith('/')) return true
return false
}
const isPdf = (name, contentType) => {
if (contentType === 'application/pdf') return true
return false
}
const isZip = (name, contentType) => {
if (!contentType || !contentType.includes('/')) return false
if (contentType.split('/')[1].includes('zip')) return true
return false
}
const isCode = (name, contentType) => {
const codeExt = ['c', 'cpp', 'go', 'py', 'java', 'rb', 'js', 'pl', 'fs',
'php', 'css', 'less', 'scss', 'coffee', 'net', 'html',
'rs', 'exs', 'scala', 'hs', 'clj', 'el', 'scm', 'lisp',
'asp', 'aspx']
const ext = name.split('.').reverse()[0]
for (var i in codeExt) {
if (ext === codeExt[i]) return true
}
return false
}
const isExcel = (name, contentType) => {
if (!contentType || !contentType.includes('/')) return false
const types = ['excel', 'spreadsheet']
const subType = contentType.split('/')[1]
for (var i in types) {
if (subType.includes(types[i])) return true
}
return false
}
const isDoc = (name, contentType) => {
if (!contentType || !contentType.includes('/')) return false
const types = ['word', '.document']
const subType = contentType.split('/')[1]
for (var i in types) {
if (subType.includes(types[i])) return true
}
return false
}
const isPresentation = (name, contentType) => {
if (!contentType || !contentType.includes('/')) return false
var types = ['powerpoint', 'presentation']
const subType = contentType.split('/')[1]
for (var i in types) {
if (subType.includes(types[i])) return true
}
return false
}
const typeToIcon = (type) => {
return (name, contentType) => {
if (!contentType || !contentType.includes('/')) return false
if (contentType.split('/')[0] === type) return true
return false
}
}
export const getDataType = (name, contentType) => {
if (contentType === "") {
contentType = mimedb.lookup(name) || 'application/octet-stream'
}
const check = [
['folder', isFolder],
['code', isCode],
['audio', typeToIcon('audio')],
['image', typeToIcon('image')],
['video', typeToIcon('video')],
['text', typeToIcon('text')],
['pdf', isPdf],
['zip', isZip],
['excel', isExcel],
['doc', isDoc],
['presentation', isPresentation]
]
for (var i in check) {
if (check[i][1](name, contentType)) return check[i][0]
}
return 'other'
}

206
browser/app/js/reducers.js Normal file
View File

@@ -0,0 +1,206 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import * as actions from './actions'
import { minioBrowserPrefix } from './constants'
export default (state = {
buckets: [],
visibleBuckets: [],
objects: [],
istruncated: true,
storageInfo: {},
serverInfo: {},
currentBucket: '',
currentPath: '',
showMakeBucketModal: false,
uploads: {},
alert: {
show: false,
type: 'danger',
message: ''
},
loginError: false,
sortNameOrder: false,
sortSizeOrder: false,
sortDateOrder: false,
latestUiVersion: currentUiVersion,
sideBarActive: false,
loginRedirectPath: minioBrowserPrefix,
settings: {
accessKey: '',
secretKey: '',
secretKeyVisible: false
},
showSettings: false,
policies: [],
deleteConfirmation: {
object: '',
show: false
},
shareObject: {
show: false,
url: '',
object: ''
},
prefixWritable: false,
checkedObjects: []
}, action) => {
let newState = Object.assign({}, state)
switch (action.type) {
case actions.SET_WEB:
newState.web = action.web
break
case actions.SET_BUCKETS:
newState.buckets = action.buckets
break
case actions.ADD_BUCKET:
newState.buckets = [action.bucket, ...newState.buckets]
newState.visibleBuckets = [action.bucket, ...newState.visibleBuckets]
break
case actions.SET_VISIBLE_BUCKETS:
newState.visibleBuckets = action.visibleBuckets
break
case actions.SET_CURRENT_BUCKET:
newState.currentBucket = action.currentBucket
break
case actions.APPEND_OBJECTS:
newState.objects = [...newState.objects, ...action.objects]
newState.marker = action.marker
newState.istruncated = action.istruncated
break
case actions.SET_OBJECTS:
newState.objects = [...action.objects]
break
case actions.RESET_OBJECTS:
newState.objects = []
newState.marker = ""
newState.istruncated = false
break
case actions.SET_CURRENT_PATH:
newState.currentPath = action.currentPath
break
case actions.SET_STORAGE_INFO:
newState.storageInfo = action.storageInfo
break
case actions.SET_SERVER_INFO:
newState.serverInfo = action.serverInfo
break
case actions.SHOW_MAKEBUCKET_MODAL:
newState.showMakeBucketModal = action.showMakeBucketModal
break
case actions.UPLOAD_PROGRESS:
newState.uploads = Object.assign({}, newState.uploads)
newState.uploads[action.slug].loaded = action.loaded
break
case actions.ADD_UPLOAD:
newState.uploads = Object.assign({}, newState.uploads, {
[action.slug]: {
loaded: 0,
size: action.size,
xhr: action.xhr,
name: action.name
}
})
break
case actions.STOP_UPLOAD:
newState.uploads = Object.assign({}, newState.uploads)
delete newState.uploads[action.slug]
break
case actions.SET_ALERT:
if (newState.alert.alertTimeout) clearTimeout(newState.alert.alertTimeout)
if (!action.alert.show) {
newState.alert = Object.assign({}, newState.alert, {
show: false
})
} else {
newState.alert = action.alert
}
break
case actions.SET_LOGIN_ERROR:
newState.loginError = true
break
case actions.SET_SHOW_ABORT_MODAL:
newState.showAbortModal = action.showAbortModal
break
case actions.SHOW_ABOUT:
newState.showAbout = action.showAbout
break
case actions.SET_SORT_NAME_ORDER:
newState.sortNameOrder = action.sortNameOrder
break
case actions.SET_SORT_SIZE_ORDER:
newState.sortSizeOrder = action.sortSizeOrder
break
case actions.SET_SORT_DATE_ORDER:
newState.sortDateOrder = action.sortDateOrder
break
case actions.SET_LATEST_UI_VERSION:
newState.latestUiVersion = action.latestUiVersion
break
case actions.SET_SIDEBAR_STATUS:
newState.sidebarStatus = action.sidebarStatus
break
case actions.SET_LOGIN_REDIRECT_PATH:
newState.loginRedirectPath = action.path
case actions.SET_LOAD_BUCKET:
newState.loadBucket = action.loadBucket
break
case actions.SET_LOAD_PATH:
newState.loadPath = action.loadPath
break
case actions.SHOW_SETTINGS:
newState.showSettings = action.showSettings
break
case actions.SET_SETTINGS:
newState.settings = Object.assign({}, newState.settings, action.settings)
break
case actions.SHOW_BUCKET_POLICY:
newState.showBucketPolicy = action.showBucketPolicy
break
case actions.SET_POLICIES:
newState.policies = action.policies
break
case actions.DELETE_CONFIRMATION:
newState.deleteConfirmation = Object.assign({}, action.payload)
break
case actions.SET_SHARE_OBJECT:
newState.shareObject = Object.assign({}, action.shareObject)
break
case actions.SET_PREFIX_WRITABLE:
newState.prefixWritable = action.prefixWritable
break
case actions.REMOVE_OBJECT:
let idx = newState.objects.findIndex(object => object.name === action.object)
if (idx == -1) break
newState.objects = [...newState.objects.slice(0, idx), ...newState.objects.slice(idx + 1)]
break
case actions.CHECKED_OBJECTS_ADD:
newState.checkedObjects = [...newState.checkedObjects, action.objectName]
break
case actions.CHECKED_OBJECTS_REMOVE:
let index = newState.checkedObjects.indexOf(action.objectName)
if (index == -1) break
newState.checkedObjects = [...newState.checkedObjects.slice(0, index), ...newState.checkedObjects.slice(index + 1)]
break
case actions.CHECKED_OBJECTS_RESET:
newState.checkedObjects = []
break
}
return newState
}

85
browser/app/js/utils.js Normal file
View File

@@ -0,0 +1,85 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { minioBrowserPrefix } from './constants.js'
export const sortObjectsByName = (objects, order) => {
let folders = objects.filter(object => object.name.endsWith('/'))
let files = objects.filter(object => !object.name.endsWith('/'))
folders = folders.sort((a, b) => {
if (a.name.toLowerCase() < b.name.toLowerCase()) return -1
if (a.name.toLowerCase() > b.name.toLowerCase()) return 1
return 0
})
files = files.sort((a, b) => {
if (a.name.toLowerCase() < b.name.toLowerCase()) return -1
if (a.name.toLowerCase() > b.name.toLowerCase()) return 1
return 0
})
if (order) {
folders = folders.reverse()
files = files.reverse()
}
return [...folders, ...files]
}
export const sortObjectsBySize = (objects, order) => {
let folders = objects.filter(object => object.name.endsWith('/'))
let files = objects.filter(object => !object.name.endsWith('/'))
files = files.sort((a, b) => a.size - b.size)
if (order)
files = files.reverse()
return [...folders, ...files]
}
export const sortObjectsByDate = (objects, order) => {
let folders = objects.filter(object => object.name.endsWith('/'))
let files = objects.filter(object => !object.name.endsWith('/'))
files = files.sort((a, b) => new Date(a.lastModified).getTime() - new Date(b.lastModified).getTime())
if (order)
files = files.reverse()
return [...folders, ...files]
}
export const pathSlice = (path) => {
path = path.replace(minioBrowserPrefix, '')
let prefix = ''
let bucket = ''
if (!path) return {
bucket,
prefix
}
let objectIndex = path.indexOf('/', 1)
if (objectIndex == -1) {
bucket = path.slice(1)
return {
bucket,
prefix
}
}
bucket = path.slice(1, objectIndex)
prefix = path.slice(objectIndex + 1)
return {
bucket,
prefix
}
}
export const pathJoin = (bucket, prefix) => {
if (!prefix)
prefix = ''
return minioBrowserPrefix + '/' + bucket + '/' + prefix
}

124
browser/app/js/web.js Normal file
View File

@@ -0,0 +1,124 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { browserHistory } from 'react-router'
import JSONrpc from './jsonrpc'
import * as actions from './actions'
import { minioBrowserPrefix } from './constants.js'
import Moment from 'moment'
import storage from 'local-storage-fallback'
export default class Web {
constructor(endpoint, dispatch) {
const namespace = 'Web'
this.dispatch = dispatch
this.JSONrpc = new JSONrpc({
endpoint,
namespace
})
}
makeCall(method, options) {
return this.JSONrpc.call(method, {
params: options
}, storage.getItem('token'))
.catch(err => {
if (err.status === 401) {
storage.removeItem('token')
browserHistory.push(`${minioBrowserPrefix}/login`)
throw new Error('Please re-login.')
}
if (err.status)
throw new Error(`Server returned error [${err.status}]`)
throw new Error('Minio server is unreachable')
})
.then(res => {
let json = JSON.parse(res.text)
let result = json.result
let error = json.error
if (error) {
throw new Error(error.message)
}
if (!Moment(result.uiVersion).isValid()) {
throw new Error("Invalid UI version in the JSON-RPC response")
}
if (result.uiVersion !== currentUiVersion
&& currentUiVersion !== 'MINIO_UI_VERSION') {
storage.setItem('newlyUpdated', true)
location.reload()
}
return result
})
}
LoggedIn() {
return !!storage.getItem('token')
}
Login(args) {
return this.makeCall('Login', args)
.then(res => {
storage.setItem('token', `${res.token}`)
return res
})
}
Logout() {
storage.removeItem('token')
}
ServerInfo() {
return this.makeCall('ServerInfo')
}
StorageInfo() {
return this.makeCall('StorageInfo')
}
ListBuckets() {
return this.makeCall('ListBuckets')
}
MakeBucket(args) {
return this.makeCall('MakeBucket', args)
}
ListObjects(args) {
return this.makeCall('ListObjects', args)
}
PresignedGet(args) {
return this.makeCall('PresignedGet', args)
}
PutObjectURL(args) {
return this.makeCall('PutObjectURL', args)
}
RemoveObject(args) {
return this.makeCall('RemoveObject', args)
}
GetAuth() {
return this.makeCall('GetAuth')
}
GenerateAuth() {
return this.makeCall('GenerateAuth')
}
SetAuth(args) {
return this.makeCall('SetAuth', args)
.then(res => {
storage.setItem('token', `${res.token}`)
return res
})
}
GetBucketPolicy(args) {
return this.makeCall('GetBucketPolicy', args)
}
SetBucketPolicy(args) {
return this.makeCall('SetBucketPolicy', args)
}
ListAllBucketPolicies(args) {
return this.makeCall('ListAllBucketPolicies', args)
}
}

View File

@@ -0,0 +1,68 @@
.alert {
border: 0;
position: fixed;
max-width: 500px;
margin: 0;
box-shadow: 0 4px 5px rgba(0, 0, 0, 0.1);
color: @white;
width: 100%;
right: 20px;
border-radius: 3px;
padding: 17px 50px 17px 17px;
z-index: 10010;
.animation-duration(800ms);
.animation-fill-mode(both);
&:not(.progress) {
top: 20px;
@media(min-width: (@screen-sm-min)) {
left: 50%;
margin-left: -250px;
}
}
&.progress {
bottom: 20px;
right: 20px;
}
&.alert-danger {
background: @red;
}
&.alert-success {
background: @green;
}
&.alert-info {
background: @blue;
}
@media(max-width: (@screen-xs-max)) {
left: 20px;
width: ~"calc(100% - 40px)";
max-width: 100%;
}
.progress {
margin: 10px 10px 8px 0;
height: 5px;
box-shadow: none;
border-radius: 1px;
background-color: @blue;
border-radius: 2px;
overflow: hidden;
}
.progress-bar {
box-shadow: none;
background-color: @white;
height: 100%;
}
.close {
position: absolute;
top: 15px;
}
}

View File

@@ -0,0 +1,13 @@
.animated{
&.infinite {
.animation-iteration-count(infinite);
}
}
@import 'fadeIn';
@import 'fadeInDown';
@import 'fadeInUp';
@import 'fadeOut';
@import 'fadeOutDown';
@import 'fadeOutUp';
@import 'zoomIn';

View File

@@ -0,0 +1,26 @@
@-webkit-keyframes fadeIn {
0% {opacity: 0;}
100% {opacity: 1;}
}
@-moz-keyframes fadeIn {
0% {opacity: 0;}
100% {opacity: 1;}
}
@-o-keyframes fadeIn {
0% {opacity: 0;}
100% {opacity: 1;}
}
@keyframes fadeIn {
0% {opacity: 0;}
100% {opacity: 1;}
}
.fadeIn {
-webkit-animation-name: fadeIn;
-moz-animation-name: fadeIn;
-o-animation-name: fadeIn;
animation-name: fadeIn;
}

View File

@@ -0,0 +1,54 @@
@-webkit-keyframes fadeInDown {
0% {
opacity: 0;
-webkit-transform: translateY(-20px);
}
100% {
opacity: 1;
-webkit-transform: translateY(0);
}
}
@-moz-keyframes fadeInDown {
0% {
opacity: 0;
-moz-transform: translateY(-20px);
}
100% {
opacity: 1;
-moz-transform: translateY(0);
}
}
@-o-keyframes fadeInDown {
0% {
opacity: 0;
-ms-transform: translateY(-20px);
}
100% {
opacity: 1;
-ms-transform: translateY(0);
}
}
@keyframes fadeInDown {
0% {
opacity: 0;
transform: translateY(-20px);
}
100% {
opacity: 1;
transform: translateY(0);
}
}
.fadeInDown {
-webkit-animation-name: fadeInDown;
-moz-animation-name: fadeInDown;
-o-animation-name: fadeInDown;
animation-name: fadeInDown;
}

View File

@@ -0,0 +1,54 @@
@-webkit-keyframes fadeInUp {
0% {
opacity: 0;
-webkit-transform: translateY(20px);
}
100% {
opacity: 1;
-webkit-transform: translateY(0);
}
}
@-moz-keyframes fadeInUp {
0% {
opacity: 0;
-moz-transform: translateY(20px);
}
100% {
opacity: 1;
-moz-transform: translateY(0);
}
}
@-o-keyframes fadeInUp {
0% {
opacity: 0;
-o-transform: translateY(20px);
}
100% {
opacity: 1;
-o-transform: translateY(0);
}
}
@keyframes fadeInUp {
0% {
opacity: 0;
transform: translateY(20px);
}
100% {
opacity: 1;
transform: translateY(0);
}
}
.fadeInUp {
-webkit-animation-name: fadeInUp;
-moz-animation-name: fadeInUp;
-o-animation-name: fadeInUp;
animation-name: fadeInUp;
}

View File

@@ -0,0 +1,26 @@
@-webkit-keyframes fadeOut {
0% {opacity: 1;}
100% {opacity: 0;}
}
@-moz-keyframes fadeOut {
0% {opacity: 1;}
100% {opacity: 0;}
}
@-o-keyframes fadeOut {
0% {opacity: 1;}
100% {opacity: 0;}
}
@keyframes fadeOut {
0% {opacity: 1;}
100% {opacity: 0;}
}
.fadeOut {
-webkit-animation-name: fadeOut;
-moz-animation-name: fadeOut;
-o-animation-name: fadeOut;
animation-name: fadeOut;
}

View File

@@ -0,0 +1,54 @@
@-webkit-keyframes fadeOutDown {
0% {
opacity: 1;
-webkit-transform: translateY(0);
}
100% {
opacity: 0;
-webkit-transform: translateY(20px);
}
}
@-moz-keyframes fadeOutDown {
0% {
opacity: 1;
-moz-transform: translateY(0);
}
100% {
opacity: 0;
-moz-transform: translateY(20px);
}
}
@-o-keyframes fadeOutDown {
0% {
opacity: 1;
-o-transform: translateY(0);
}
100% {
opacity: 0;
-o-transform: translateY(20px);
}
}
@keyframes fadeOutDown {
0% {
opacity: 1;
transform: translateY(0);
}
100% {
opacity: 0;
transform: translateY(20px);
}
}
.fadeOutDown {
-webkit-animation-name: fadeOutDown;
-moz-animation-name: fadeOutDown;
-o-animation-name: fadeOutDown;
animation-name: fadeOutDown;
}

View File

@@ -0,0 +1,51 @@
@-webkit-keyframes fadeOutUp {
0% {
opacity: 1;
-webkit-transform: translateY(0);
}
100% {
opacity: 0;
-webkit-transform: translateY(-20px);
}
}
@-moz-keyframes fadeOutUp {
0% {
opacity: 1;
-moz-transform: translateY(0);
}
100% {
opacity: 0;
-moz-transform: translateY(-20px);
}
}
@-o-keyframes fadeOutUp {
0% {
opacity: 1;
-o-transform: translateY(0);
}
100% {
opacity: 0;
-o-transform: translateY(-20px);
}
}
@keyframes fadeOutUp {
0% {
opacity: 1;
transform: translateY(0);
}
100% {
opacity: 0;
transform: translateY(-20px);
}
}
.fadeOutUp {
-webkit-animation-name: fadeOutUp;
-moz-animation-name: fadeOutUp;
-o-animation-name: fadeOutUp;
animation-name: fadeOutUp;
}

View File

@@ -0,0 +1,23 @@
@-webkit-keyframes zoomIn {
from {
opacity: 0;
-webkit-transform: scale3d(.3, .3, .3);
transform: scale3d(.3, .3, .3);
}
50% {
opacity: 1;
}
}
@keyframes zoomIn {
from {
opacity: 0;
-webkit-transform: scale3d(.3, .3, .3);
transform: scale3d(.3, .3, .3);
}
50% {
opacity: 1;
}
}

View File

@@ -0,0 +1,31 @@
* {
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
&:focus,
&:active {
outline: 0;
}
}
html {
font-size: 10px;
-webkit-tap-highlight-color: rgba(0,0,0,0);
}
html,
body {
min-height: 100%;
}
a {
.transition(color);
.transition-duration(300ms);
}
button {
border: 0;
}

View File

@@ -0,0 +1,57 @@
.btn {
border: 0;
padding: 5px 10px;
font-size: 12px;
line-height: 1.5;
border-radius: 2px;
text-align: center;
.transition(all);
.transition-duration(300ms);
&:hover,
&:focus {
.opacity(0.9);
}
}
/*-----------------------------------
Button Variants
------------------------------------*/
.btn-variant(@bg-color, @color) {
color: @color;
background-color: @bg-color;
&:hover,
&:focus {
color: @color;
background-color: darken(@bg-color, 6%);
}
}
.btn-block {
display: block;
width: 100%;
}
.btn-white {
.btn-variant(#fff, darken(@text-color, 20%));
}
.btn-link {
.btn-variant(#eee, #545454);
}
.btn-danger {
.btn-variant(@red, @white);
}
.btn-primary {
.btn-variant(@blue, @white);
}
.btn-success {
.btn-variant(@green, @white);
}
//-----------------------------------

View File

@@ -0,0 +1,26 @@
.dropdown-menu {
padding: 15px 0;
top: 0;
margin-top: -1px;
& > li {
& > a {
padding: 8px 20px;
font-size: 15px;
& > i {
width: 20px;
position: relative;
top: 1px;
}
}
}
}
.dropdown-menu-right {
& > li {
& > a {
text-align: right;
}
}
}

View File

@@ -0,0 +1,160 @@
/*------------------------------
Layout
--------------------------------*/
.file-explorer {
background-color: @white;
position: relative;
height: 100%;
&.toggled {
height: 100vh;
overflow: hidden;
}
}
.fe-body {
@media(min-width: @screen-md-min) {
padding: 0 0 40px @fe-sidebar-width;
}
@media(max-width: @screen-sm-max) {
padding: 75px 0 80px;
}
min-height:100vh;
overflow: auto;
}
/*------------------------------
Create and Upload Button
--------------------------------*/
.feb-actions {
position: fixed;
bottom: 30px;
right: 30px;
.dropdown-menu {
min-width: 55px;
width: 55px;
text-align: center;
background: transparent;
box-shadow: none;
margin: 0;
}
&.open {
.feba-btn {
.scale(1);
&:first-child {
.animation-name(feba-btn-anim);
.animation-duration(300ms);
}
&:last-child {
.animation-name(feba-btn-anim);
.animation-duration(100ms);
}
}
.feba-toggle {
background: darken(@red, 10%);
& > span {
.rotate(135deg);
}
}
}
}
.feba-toggle {
width: 55px;
height: 55px;
line-height: 55px;
border-radius: 50%;
background: @red;
box-shadow: 0 2px 3px rgba(0, 0, 0, 0.15);
display: inline-block;
text-align: center;
border: 0;
padding: 0;
span {
display: inline-block;
height: 100%;
width: 100%;
}
i {
color: @white;
font-size: 17px;
line-height: 58px;
}
}
.feba-toggle,
.feba-toggle > span {
.transition(all);
.transition-duration(250ms);
.backface-visibility(hidden);
}
.feba-btn {
width: 40px;
margin-top: 10px;
height: 40px;
border-radius: 50%;
text-align: center;
display: inline-block;
color: @white;
line-height: 40px;
box-shadow: 0 2px 3px rgba(0, 0, 0, 0.15);
-webkit-transform: scale(0);
transform: scale(0);
position: relative;
&:hover,
&:focus {
color: @white;
}
label {
width: 100%;
height: 100%;
position: absolute;
left: 0;
top: 0;
cursor: pointer;
}
}
.feba-bucket {
background: @orange;
}
.feba-upload {
background: @yellow;
}
@-webkit-keyframes feba-btn-anim {
from {
.scale(0);
.opacity(0);
}
to {
.scale(1);
.opacity(1);
}
}
@keyframes feba-btn-anim {
from {
.scale(0);
.opacity(0);
}
to {
.scale(1);
.opacity(1);
}
}

View File

@@ -0,0 +1,7 @@
@font-face {
font-family: Lato;
src: url('../../fonts/lato/lato-normal.woff2') format('woff2'),
url('../../fonts/lato/lato-normal.woff') format('woff');
font-weight: normal;
font-style: normal;
}

View File

@@ -0,0 +1,262 @@
.form-control {
border: 0;
border-bottom: 1px solid @input-border;
color: #32393F;
padding: 5px;
width: 100%;
font-size: 13px;
background-color: transparent;
}
select.form-control {
-webkit-appearance: none;
-moz-appearance: none;
border-radius: 0;
background: url(../../img/select-caret.svg) no-repeat bottom 7px right;
}
/*--------------------------
Input Group
----------------------------*/
.input-group {
position: relative;
&:not(:last-child) {
margin-bottom: 25px;
}
label:not(.ig-label) {
font-size: 13px;
display: block;
margin-bottom: 10px;
}
}
.ig-label {
position: absolute;
text-align: center;
bottom: 7px;
left: 0;
width: 100%;
.transition(all);
.transition-duration(250ms);
padding: 2px 0 3px;
border-radius: 2px;
font-weight: 400;
}
.ig-helpers {
z-index: 1;
width: 100%;
left: 0;
&,
&:before,
&:after {
position: absolute;
height: 2px;
bottom: 0;
}
&:before,
&:after {
content: '';
width: 0;
.transition(all);
.transition-duration(250ms);
background-color: #03A9F4;
}
&:before {
left: 50%;
}
&:after {
right: 50%;
}
}
.ig-text {
width: 100%;
height: 40px;
border: 0;
background: transparent !important;
text-align: center;
position: relative;
z-index: 1;
border-bottom: 1px solid #eee;
color: #32393F;
font-size: 13px;
&:focus + .ig-helpers {
&:before,
&:after {
width: 50%;
}
}
&:valid,
&:disabled,
&:focus {
& ~ .ig-label {
bottom: 35px;
font-size: 13px;
z-index: 1;
}
}
&:disabled {
.opacity(0.5);
}
}
.ig-dark {
.ig-text {
color: @white !important;
border-color: rgba(255,255,255,0.1) !important;
}
.ig-helpers {
&:before,
&:after {
background-color: #dfdfdf;
height: 1px;
}
}
}
.ig-left {
.ig-label,
.ig-text {
text-align: left;
}
}
.ig-error {
.ig-label {
color: #E23F3F;
}
.ig-helpers i {
&:first-child,
&:first-child:before,
&:first-child:after {
background: rgba(226, 63, 63, 0.43);
}
&:last-child,
&:last-child:before,
&:last-child:after {
background: #E23F3F !important;
}
}
&:after {
content: "\f05a";
font-family: FontAwesome;
position: absolute;
top: 17px;
right: 9px;
font-size: 20px;
color: #D33D3E;
}
}
.ig-search {
&:before {
font-family: @font-family-icon;
content: '\f002';
font-size: 15px;
position: absolute;
left: 2px;
top: 8px;
}
.ig-text {
padding-left: 25px;
}
}
/*--------------------------
Share Spinners
----------------------------*/
.set-expire {
border: 1px solid @input-border;
margin: 35px 0 30px;
position: relative;
&:before {
content: '';
position: absolute;
left: 0;
top: 0;
width: 100%;
height: 100%;
z-index: 1;
}
}
.set-expire-item {
padding: 9px 5px 3px;
position: relative;
display: table-cell;
width: 1%;
text-align: center;
.user-select(none);
&:not(:last-child) {
border-right: 1px solid @input-border;
}
}
.set-expire-title {
font-size: 10px;
text-transform: uppercase;
}
.set-expire-value {
display: inline-block;
overflow: hidden;
position: relative;
left: -8px;
input {
.user-select(none);
font-size: 20px;
text-align: center;
position: relative;
right: -15px;
border: 0;
color: @text-strong-color;
padding: 0;
height: 25px;
width: 100%;
font-weight: normal;
}
}
.set-expire-decrease,
.set-expire-increase {
position: absolute;
width: 20px;
height: 20px;
background: url(../../img/arrow.svg) no-repeat center;
background-size: 85%;
left: 50%;
margin-left: -10px;
.opacity(0.2);
cursor: pointer;
&:hover {
.opacity(0.5);
}
}
.set-expire-increase {
top: -25px;
}
.set-expire-decrease {
bottom: -27px;
.rotate(-180deg);
}

View File

@@ -0,0 +1,83 @@
/*----------------------------
Text Alignment
-----------------------------*/
.text-center { text-align: center !important; }
.text-left { text-align: left !important; }
.text-right { text-align: right !important; }
/*----------------------------
Float
-----------------------------*/
.clearfix { .clearfix(); }
.pull-right { float: right !important; }
.pull-left { float: left !important; }
/*----------------------------
Position
-----------------------------*/
.p-relative { position: relative; }
/*---------------------------------------------------------------------------
Generate Margin Class
margin, margin-top, margin-bottom, margin-left, margin-right
----------------------------------------------------------------------------*/
.margin (@label, @size: 1, @key:1) when (@size =< 30){
.m-@{key} {
margin: @size !important;
}
.m-t-@{key} {
margin-top: @size !important;
}
.m-b-@{key} {
margin-bottom: @size !important;
}
.m-l-@{key} {
margin-left: @size !important;
}
.m-r-@{key} {
margin-right: @size !important;
}
.margin(@label - 5; @size + 5; @key + 5);
}
.margin(25, 0px, 0);
/*---------------------------------------------------------------------------
Generate Padding Class
padding, padding-top, padding-bottom, padding-left, padding-right
----------------------------------------------------------------------------*/
.padding (@label, @size: 1, @key:1) when (@size =< 30){
.p-@{key} {
padding: @size !important;
}
.p-t-@{key} {
padding-top: @size !important;
}
.p-b-@{key} {
padding-bottom: @size !important;
}
.p-l-@{key} {
padding-left: @size !important;
}
.p-r-@{key} {
padding-right: @size !important;
}
.padding(@label - 5; @size + 5; @key + 5);
}
.padding(25, 0px, 0);

View File

@@ -0,0 +1,240 @@
/*--------------------------
Header
----------------------------*/
.fe-header {
@media(min-width: (@screen-sm-min - 100)) {
position: relative;
padding: 40px 40px 20px 45px;
}
@media(max-width: (@screen-xs-max - 100)) {
padding: 20px;
}
h2 {
font-size: 16px;
font-weight: normal;
margin: 0;
& > span {
margin-bottom: 7px;
display: inline-block;
&:not(:first-child) {
&:before {
content: '/';
margin: 0 4px;
color: @text-color;
}
}
}
}
p {
margin-top: 7px;
}
}
/*--------------------------
Disk usage
----------------------------*/
.feh-usage {
margin-top: 12px;
max-width: 285px;
@media(max-width: (@screen-xs-max - 100px)) {
max-width: 100%;
font-size: 12px;
}
& > ul {
margin-top: 7px;
list-style: none;
padding: 0;
& > li {
padding-right: 0;
display: inline-block;
}
}
}
.fehu-chart {
height: 5px;
background: #eee;
position: relative;
border-radius: 2px;
overflow: hidden;
& > div {
position: absolute;
left: 0;
height: 100%;
background: @link-color;
}
}
/*--------------------------
Header Actions
----------------------------*/
.feh-actions {
list-style: none;
padding: 0;
margin: 0;
position: absolute;
right: 35px;
top: 30px;
z-index: 11;
@media(max-width: (@screen-sm-max)) {
top: 7px;
right: 10px;
position: fixed;
}
& > li {
display: inline-block;
text-align: right;
vertical-align: top;
line-height: 100%;
& > a,
& > .btn-group > button {
display: block;
height: 45px;
min-width: 45px;
text-align: center;
border-radius: 50%;
padding: 0;
border: 0;
background: none;
@media(min-width: @screen-md-min) {
color: #7B7B7B;
font-size: 21px;
line-height: 45px;
.transition(all);
.transition-duration(300ms);
&:hover {
background: rgba(0,0,0,0.09);
}
}
@media(max-width: (@screen-sm-max)) {
background: url(../../img/more-h-light.svg) no-repeat center;
.fa-reorder {
display: none;
}
}
}
}
}
/*--------------------------
Mobile Header
----------------------------*/
@media(max-width: @screen-sm-max) {
.fe-header-mobile {
background-color: @dark-gray;
padding: 10px 50px 9px 12px;
text-align: center;
position: fixed;
z-index: 10;
box-shadow: 0 0 10px rgba(0,0,0,0.3);
left: 0;
top: 0;
width: 100%;
.mh-logo {
height: 35px;
position: relative;
top: 4px;
}
}
.feh-trigger {
width: 41px;
height: 41px;
cursor: pointer;
float: left;
position: relative;
text-align: center;
&:before,
&:after {
content: "";
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
border-radius: 50%;
}
&:after {
z-index: 1;
}
&:before {
background: rgba(255, 255, 255, 0.1);
.transition(all);
.transition-duration(300ms);
.scale(0);
}
}
.feht-toggled {
&:before {
.scale(1);
}
.feht-lines {
.rotate(180deg);
& > div {
&.top {
width: 12px;
transform: translateX(8px) translateY(1px) rotate(45deg);
-webkit-transform: translateX(8px) translateY(1px) rotate(45deg);
}
&.bottom {
width: 12px;
transform: translateX(8px) translateY(-1px) rotate(-45deg);
-webkit-transform: translateX(8px) translateY(-1px) rotate(-45deg);
}
}
}
}
.feht-lines,
.feht-lines > div {
.transition(all);
.transition-duration(300ms);
}
.feht-lines {
width: 18px;
height: 12px;
display: inline-block;
margin-top: 14px;
& > div {
background-color: #EAEAEA;
width: 18px;
height: 2px;
&.center {
margin: 3px 0;
}
}
}
}

View File

@@ -0,0 +1,81 @@
.ie-warning {
background-color: #ff5252;
width: 100%;
height: 100%;
position: fixed;
left: 0;
top: 0;
text-align: center;
&:before {
width: 1px;
content: '';
height: 100%;
}
&:before,
.iw-inner {
display: inline-block;
vertical-align: middle;
}
}
.iw-inner {
width: 470px;
height: 300px;
background-color: @white;
border-radius: 5px;
padding: 40px;
position: relative;
ul {
list-style: none;
padding: 0;
margin: 0;
width: 230px;
margin-left: 80px;
margin-top: 16px;
& > li {
float: left;
& > a {
display: block;
padding: 10px 15px 7px;
font-size: 14px;
margin: 0 1px;
border-radius: 3px;
&:hover {
background: #eee;
}
img {
height: 40px;
margin-bottom: 5px;
}
}
}
}
}
.iwi-icon {
color: #ff5252;
font-size: 40px;
display: block;
line-height: 100%;
margin-bottom: 15px;
}
.iwi-skip {
position: absolute;
left: 0;
bottom: -35px;
width: 100%;
color: rgba(255, 255, 255, 0.6);
cursor: pointer;
&:hover {
color: @white;
}
}

View File

@@ -0,0 +1,486 @@
/*--------------------------
Row
----------------------------*/
.fesl-row {
position: relative;
@media (min-width: (@screen-sm-min - 100px)) {
padding: 5px 20px 5px 40px;
display: flex;
flex-flow: row nowrap;
justify-content: space-between;
}
@media(max-width: (@screen-xs-max - 100px)) {
padding: 5px 20px;
}
.clearfix();
}
header.fesl-row {
@media (min-width:(@screen-sm-min - 100px)) {
margin-bottom: 20px;
border-bottom: 1px solid lighten(@text-muted-color, 20%);
padding-left: 30px;
.fesl-item,
.fesli-sort {
.transition(all);
.transition-duration(300ms);
}
.fesl-item {
cursor: pointer;
color: @text-color;
font-weight: 500;
margin-bottom: -5px;
& > .fesli-sort {
float: right;
margin: 4px 0 0;
.opacity(0);
color: @dark-gray;
font-size: 14px;
}
&:hover:not(.fesl-item-actions) {
background: lighten(@text-muted-color, 22%);
color: @dark-gray;
& > .fesli-sort {
.opacity(0.5);
}
}
}
}
@media (max-width:(@screen-xs-max - 100px)) {
display: none;
}
}
.list-type(@background, @icon) {
.fis-icon {
background-color: @background;
&:before {
content: @icon;
}
}
}
div.fesl-row {
border-bottom: 1px solid transparent;
cursor: default;
.transition(background-color);
.transition-duration(500ms);
@media (max-width: (@screen-xs-max - 100px)) {
padding: 5px 20px;
}
&:not(.fesl-row-selected) {
&:nth-child(even) {
background-color: @list-row-even-bg;
}
}
&:hover {
.fis-icon {
&:before {
.opacity(0)
}
}
.fis-helper {
&:before {
.opacity(1);
}
}
}
/*--------------------------
Icons
----------------------------*/
&[data-type=folder] { .list-type(#a1d6dd, '\f114'); }
&[data-type=pdf] {.list-type(#fa7775, '\f1c1'); }
&[data-type=zip] { .list-type(#427089, '\f1c6'); }
&[data-type=audio] { .list-type(#009688, '\f1c7'); }
&[data-type=code] { .list-type(#997867, "\f1c9"); }
&[data-type=excel] { .list-type(#f1c3, '\f1c3'); }
&[data-type=image] { .list-type(#f06292, '\f1c5'); }
&[data-type=video] { .list-type(#f8c363, '\f1c8'); }
&[data-type=other] { .list-type(#afafaf, '\f016'); }
&[data-type=text] { .list-type(#8a8a8a, '\f0f6'); }
&[data-type=doc] { .list-type(#2196f5, '\f1c2'); }
&[data-type=presentation] { .list-type(#896ea6, '\f1c4'); }
&.fesl-loading{
&:before {
content: '';
}
&:after {
.list-loader(20px, 20px, rgba(255, 255, 255, 0.5), @white);
left: 57px;
top: 17px;
@media (max-width: (@screen-xs-max - 100px)) {
left: 27px;
}
}
}
}
.fesl-row-selected {
background-color: @list-row-selected-bg;
&, .fesl-item a {
color: darken(@text-color, 10%);
}
}
.fi-select {
float: left;
position: relative;
width: 35px;
height: 35px;
margin: 3px 0;
@media(max-width: (@screen-xs-max - 100px)) {
margin-right: 15px;
}
input {
position: absolute;
left: 0;
top: 0;
width: 35px;
height: 35px;
z-index: 20;
opacity: 0;
cursor: pointer;
&:checked {
& ~ .fis-icon {
background-color: #32393F;
&:before {
opacity: 0;
}
}
& ~ .fis-helper {
&:before {
.scale(0);
}
&:after {
.scale(1);
}
}
}
}
}
.fis-icon {
display: inline-block;
vertical-align: top;
border-radius: 50%;
width: 35px;
height: 35px;
.transition(background-color);
.transition-duration(250ms);
&:before {
width: 100%;
height: 100%;
text-align: center;
position: absolute;
border-radius: 50%;
font-family: @font-family-icon;
line-height: 35px;
font-size: 16px;
color: @white;
.transition(all);
.transition-duration(300ms);
font-style: normal;
}
}
.fis-helper {
&:before,
&:after {
position: absolute;
.transition(all);
.transition-duration(250ms);
}
&:before {
content: '';
width: 15px;
height: 15px;
border: 2px solid @white;
z-index: 10;
border-radius: 2px;
top: 10px;
left: 10px;
opacity: 0;
}
&:after {
font-family: @font-family-icon;
content: '\f00c';
top: 8px;
left: 9px;
color: @white;
font-size: 14px;
.scale(0);
}
}
/*--------------------------
Files and Folders
----------------------------*/
.fesl-item {
display: block;
a {
color: darken(@text-color, 5%);
}
@media(min-width: (@screen-sm-min - 100px)) {
&:not(.fesl-item-actions):not(.fesl-item-icon) {
text-overflow: ellipsis;
padding: 10px 15px;
white-space: nowrap;
overflow: hidden;
}
&.fesl-item-name {
flex: 3;
}
&.fesl-item-size {
width: 140px;
}
&.fesl-item-modified {
width: 190px;
}
&.fesl-item-actions {
width: 40px;
}
}
@media(max-width: (@screen-xs-max - 100px)) {
padding: 0;
&.fesl-item-name {
width: 100%;
margin-bottom: 3px;
}
&.fesl-item-size,
&.fesl-item-modified {
font-size: 12px;
color: #B5B5B5;
float: left;
}
&.fesl-item-modified {
max-width: 72px;
white-space: nowrap;
overflow: hidden;
}
&.fesl-item-size {
margin-right: 10px;
}
&.fesl-item-actions {
position: absolute;
top: 5px;
right: 10px;
}
}
}
/*--------------------------
Action buttons
----------------------------*/
.fia-toggle {
height: 36px;
width: 36px;
background: transparent url(../../img/more-h.svg) no-repeat center;
position: relative;
top: 3px;
.opacity(0.4);
&:hover {
.opacity(0.7);
}
}
.fesl-item-actions {
.dropdown-menu {
background-color: transparent;
box-shadow: none;
padding: 0;
right: 38px;
left: auto;
margin: 0;
height: 100%;
text-align: right;
}
.dropdown {
&.open {
.dropdown-menu {
.fiad-action {
right: 0;
}
}
}
}
}
.fiad-action {
height: 35px;
width: 35px;
background: @amber;
display: inline-block;
border-radius: 50%;
text-align: center;
line-height: 35px;
font-weight: normal;
position: relative;
top: 4px;
margin-left: 5px;
.animation-name(fiad-action-anim);
.transform-origin(center center);
.backface-visibility(none);
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.1);
&:nth-child(2) {
.animation-duration(100ms);
}
&:nth-child(1) {
.animation-duration(250ms);
}
& > i {
font-size: 14px;
color: @white;
}
&:hover {
background-color: darken(@amber, 3%);
}
}
.list-actions {
position: fixed;
.translate3d(0, -100%, 0);
.opacity(0);
.transition(all);
.transition-duration(200ms);
padding: 20px 70px 20px 25px;
top: 0;
left: 0;
width: 100%;
background-color: @brand-primary;
z-index: 20;
box-shadow: 0 0 10px rgba(0,0,0,0.3);
text-align: center;
&.list-actions-toggled {
.translate3d(0, 0, 0);
.opacity(1);
}
}
.la-close {
position: absolute;
right: 20px;
top: 0;
color: #fff;
width: 30px;
height: 30px;
border-radius: 50%;
text-align: center;
line-height: 30px !important;
background: rgba(255, 255, 255, 0.1);
font-weight: normal;
bottom: 0;
margin: auto;
cursor: pointer;
&:hover {
background-color: rgba(255, 255, 255, 0.2);
}
}
.la-label {
color: @white;
float: left;
padding: 4px 0;
.fa {
font-size: 22px;
vertical-align: top;
margin-right: 10px;
margin-top: -1px;
}
}
.la-actions {
button {
background-color: transparent;
border: 2px solid rgba(255,255,255,0.9);
color: @white;
border-radius: 2px;
padding: 5px 10px;
font-size: 13px;
.transition(all);
.transition-duration(300ms);
margin-left: 10px;
&:hover {
background-color: @white;
color: @brand-primary;
}
}
}
@-webkit-keyframes fiad-action-anim {
from {
.scale(0);
.opacity(0);
right: -20px;
}
to {
.scale(1);
.opacity(1);
right: 0;
}
}
@keyframes fiad-action-anim {
from {
.scale(0);
.opacity(0);
right: -20px;
}
to {
.scale(1);
.opacity(1);
right: 0;
}
}

View File

@@ -0,0 +1,104 @@
.login {
height: 100vh;
min-height: 500px;
background: @dark-gray;
text-align: center;
&:before {
height: ~"calc(100% - 110px)";
width: 1px;
content: "";
}
}
.l-wrap,
.login:before {
display: inline-block;
vertical-align: middle;
}
.l-wrap {
width: 80%;
max-width: 500px;
margin-top: -50px;
&.toggled {
display: inline-block;
}
.input-group:not(:last-child) {
margin-bottom: 40px;
}
}
.l-footer {
height: 110px;
padding: 0 50px;
}
.lf-logo {
float: right;
img {
width: 40px;
}
}
.lf-server {
float: left;
color: rgba(255, 255, 255, 0.4);
font-size: 20px;
font-weight: 400;
padding-top: 40px;
}
@media (max-width: @screen-sm-min) {
.lf-logo,
.lf-server {
float: none;
display: block;
text-align: center;
width: 100%;
}
.lf-logo {
margin-bottom: 5px;
}
.lf-server {
font-size: 15px;
}
}
.lw-btn {
width: 50px;
height: 50px;
border: 1px solid @white;
display: inline-block;
border-radius: 50%;
font-size: 22px;
color: @white;
.transition(all);
.transition-duration(300ms);
opacity: 0.3;
background-color: transparent;
line-height: 45px;
padding: 0;
&:hover {
color: @white;
opacity: 0.8;
border-color: @white;
}
i {
display: block;
width: 100%;
padding-left: 3px;
}
}
/*------------------------------
Chrome autofill fix
-------------------------------*/
input:-webkit-autofill {
-webkit-box-shadow:0 0 0 50px @dark-gray inset !important;
-webkit-text-fill-color: @white !important;
}

View File

@@ -0,0 +1,116 @@
/*--------------------------
Close
----------------------------*/
.close-variant(@color, @bg-color, @color-hover, @bg-color-hover) {
span {
background-color: @bg-color;
color: @color;
}
&:hover,
&:focus {
span {
background-color: @bg-color-hover;
color: @color-hover;
}
}
}
.close {
right: 15px;
font-weight: normal;
opacity: 1;
font-size: 18px;
position: absolute;
text-align: center;
top: 16px;
z-index: 1;
padding: 0;
border: 0;
background-color: transparent;
span {
width: 25px;
height: 25px;
display: block;
border-radius: 50%;
line-height: 24px;
text-shadow: none;
}
&:not(.close-alt) {
.close-variant(rgba(255, 255, 255, 0.8), rgba(255, 255, 255, 0.1), @white, rgba(255, 255, 255, 0.2));
}
}
.close-alt {
.close-variant(#989898, #efefef, #7b7b7b, #e8e8e8);
}
/*--------------------------
Hidden
----------------------------*/
.hidden {
display: none !important;
}
/*--------------------------
Copy text
----------------------------*/
.copy-text {
input {
width: 100%;
border-radius: 1px;
border: 1px solid @input-border;
padding: 7px 12px;
font-size: 13px;
line-height: 100%;
cursor: text;
.transition(border-color);
.transition-duration(300ms);
&:hover {
border-color: darken(@input-border, 5%);
}
}
}
/*--------------------------
Sharing
----------------------------*/
.share-availability {
margin-bottom: 40px;
&:before,
&:after {
position: absolute;
bottom: -30px;
font-size: 10px;
}
&:before {
content: '01 Sec';
left: 0;
}
&:after {
content: '7 days';
right: 0;
}
}
.modal-aheader {
height: 100px;
&:before {
height: 0 !important;
}
.modal-dialog {
margin: 0;
vertical-align: top;
}
}

View File

@@ -0,0 +1,52 @@
/*--------------------------
User Select
----------------------------*/
.user-select(@value) {
-webkit-user-select: @value;
-moz-user-select: @value;
-ms-user-select: @value;
user-select: @value;
}
/*----------------------------------------
CSS Animations based on animate.css
-----------------------------------------*/
.animated(@name, @duration) {
-webkit-animation-name: @name;
animation-name: @name;
-webkit-animation-duration: @duration;
animation-duration: @duration;
-webkit-animation-fill-mode: both;
animation-fill-mode: both;
}
/*-------------------------------------------------
For loop mixin for generate custom classes
--------------------------------------------------*/
.for(@i, @n) {.-each(@i)}
.for(@n) when (isnumber(@n)) {.for(1, @n)}
.for(@i, @n) when not (@i = @n) {
.for((@i + (@n - @i) / abs(@n - @i)), @n);
}
.for(@array) when (default()) {.for-impl_(length(@array))}
.for-impl_(@i) when (@i > 1) {.for-impl_((@i - 1))}
.for-impl_(@i) when (@i > 0) {.-each(extract(@array, @i))}
/*----------------------------------------
List Loader
-----------------------------------------*/
.list-loader(@width, @height, @borderColor, @borderColorBottom) {
content: '';
width: @width;
height: @height;
border-radius: 50%;
.animated(zoomIn, 500ms);
border: 2px solid @borderColor;
border-bottom-color: @borderColorBottom;
position: absolute;
z-index: 1;
-webkit-animation: zoomIn 250ms, spin 700ms 250ms infinite linear;
animation: zoomIn 250ms, spin 700ms 250ms infinite linear;
}

View File

@@ -0,0 +1,294 @@
/*--------------------------
Modal
----------------------------*/
.modal {
@media(min-width: @screen-sm-min) {
text-align: center;
&:before {
content: '';
height: 100%;
width: 1px;
display: inline-block;
vertical-align: middle;
}
.modal-dialog {
text-align: left;
margin: 10px auto;
display: inline-block;
vertical-align: middle;
}
}
}
.modal-dark {
.modal-header {
color: rgba(255, 255, 255, 0.4);
small {
color: rgba(255, 255, 255, 0.2);
}
}
.modal-content {
background-color: @dark-gray;
}
}
.modal-backdrop {
.animated(fadeIn, 200ms);
}
.modal-dialog {
.animated(zoomIn, 200ms);
}
.modal-header {
color: @text-strong-color;
position: relative;
small {
display: block;
text-transform: none;
font-size: 12px;
margin-top: 5px;
color: #a8a8a8;
}
}
.modal-content {
border-radius: 3px;
box-shadow: none;
}
.modal-footer {
padding: 0 30px 30px;
text-align: center;
}
/*--------------------------
Dialog
----------------------------*/
.modal-confirm {
.modal-dialog {
text-align: center;
}
}
.mc-icon {
margin: 0 0 10px;
& > i {
font-size: 60px;
}
}
.mci-red {
color: #ff8f8f;
}
.mci-amber {
color: @amber;
}
.mci-green {
color: #64e096;
}
.mc-text {
color: @text-strong-color;
}
.mc-sub {
color: @text-muted-color;
margin-top: 5px;
font-size: 13px;
}
//--------------------------
/*--------------------------
About
----------------------------*/
.modal-about {
@media (max-width: @screen-xs-max) {
text-align: center;
.modal-dialog {
max-width: 400px;
width: 90%;
margin: 20px auto 0;
}
}
}
.ma-inner {
display: flex;
flex-direction: row;
align-items: center;
min-height: 350px;
position: relative;
@media (min-width: @screen-sm-min) {
&:before {
content: '';
width: 150px;
height: 100%;
top: 0;
left: 0;
position: absolute;
border-radius: 3px 0 0px 3px;
background-color: #23282C;
}
}
}
.mai-item {
&:first-child {
width: 150px;
text-align: center;
}
&:last-child {
flex: 4;
padding: 30px;
}
}
.maii-logo {
width: 70px;
position: relative;
}
.maii-list {
list-style: none;
padding: 0;
& > li {
margin-bottom: 15px;
div {
color: rgba(255, 255, 255, 0.8);
text-transform: uppercase;
font-size: 14px;
}
small {
font-size: 13px;
color: rgba(255, 255, 255, 0.4);
}
}
}
//--------------------------
/*--------------------------
Preferences
----------------------------*/
.toggle-password {
position: absolute;
bottom: 30px;
right: 35px;
width: 30px;
height: 30px;
border: 1px solid #eee;
border-radius: 0;
text-align: center;
cursor: pointer;
z-index: 10;
background-color: @white;
padding-top: 5px;
&.toggled {
background: #eee;
}
}
//--------------------------
/*--------------------------
Policy
----------------------------*/
.pm-body {
padding-bottom: 30px;
}
.pmb-header {
margin-bottom: 35px;
}
.pmb-list {
display: flex;
flex-flow: row nowrap;
align-items: center;
justify-content: center;
padding: 10px 35px;
&:nth-child(even) {
background-color: #F7F7F7;
}
.form-control {
padding-left: 0;
padding-right: 0;
}
}
header.pmb-list {
margin: 20px 0 10px;
}
.pmbl-item {
display: block;
font-size: 13px;
&:nth-child(1) {
flex: 2;
}
&:nth-child(2) {
margin: 0 25px;
width: 150px;
}
&:nth-child(3) {
width: 70px;
}
}
div.pmb-list {
select {
border: 0;
}
.pml-item {
&:not(:last-child) {
padding: 0 5px;
}
}
}
//--------------------------
/*--------------------------
Create Bucket
----------------------------*/
.modal-create-bucket {
.modal-dialog {
position: fixed;
right: 25px;
bottom: 95px;
margin: 0;
height: 110px;
}
.modal-content {
width: 100%;
height: 100%;
}
}
//--------------------------

View File

@@ -0,0 +1,187 @@
/*--------------------------
Sidebar
----------------------------*/
.fe-sidebar {
width: @fe-sidebar-width;
background-color: @dark-gray;
position: fixed;
height: 100%;
overflow: hidden;
padding: 25px;
@media(min-width: @screen-md-min) {
.translate3d(0, 0, 0);
}
@media(max-width: @screen-sm-max) {
padding-top: 85px;
z-index: 9;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.65);
.transition(all);
.transition-duration(300ms);
.translate3d((-@fe-sidebar-width - 15px), 0, 0);
&.toggled {
.translate3d(0, 0, 0);
}
}
a {
color: rgba(255, 255, 255, 0.58);
&:hover {
color: @white;
}
}
}
/*--------------------------
Header
----------------------------*/
.fes-header {
margin-bottom: 40px;
img,
h2 {
float: left;
}
h2 {
margin: 13px 0 0 10px;
font-weight: normal;
}
img {
width: 32px;
}
}
/*--------------------------
List
----------------------------*/
.fesl-inner {
height: ~"calc(100vh - 260px)";
overflow: auto;
padding: 0;
margin: 0 -25px;
& li {
position: relative;
& > a {
display: block;
padding: 10px 45px 12px 55px;
word-wrap: break-word;
&:before {
font-family: FontAwesome;
content: '\f0a0';
font-size: 17px;
position: absolute;
top: 10px;
left: 25px;
.opacity(0.8);
}
&.fesli-loading {
&:before {
.list-loader(20px, 20px, rgba(255, 255, 255, 0.1), rgba(255, 255, 255, 0.5));
left: 32px;
top: 0;
bottom: 0;
margin: auto;
}
}
}
&.active {
background-color: #282e32;
& > a {
color: @white;
}
}
&:not(.active):hover {
background-color: rgba(0, 0, 0, 0.1);
& > a {
color: @white;
}
}
&:hover {
.fesli-trigger {
.opacity(0.6);
&:hover {
.opacity(1);
}
}
}
}
ul {
list-style: none;
padding: 0;
margin: 0;
}
&:hover .scrollbar-vertical {
opacity: 1;
}
}
.fesli-trigger {
.opacity(0);
.transition(all);
.transition-duration(200ms);
position: absolute;
top: 0;
right: 0;
width: 35px;
height: 100%;
cursor: pointer;
background: url(../../img/more-h-light.svg) no-repeat top 20px left;
}
/* Scrollbar */
.scrollbar-vertical {
position: absolute;
right: 5px;
width: 4px;
height: 100%;
opacity: 0;
.transition(opacity);
.transition-duration(300ms);
div {
border-radius: 1px !important;
background-color: #6a6a6a !important;
}
}
/*--------------------------
Host
----------------------------*/
.fes-host {
position: fixed;
left: 0;
bottom: 0;
z-index: 1;
background: @dark-gray;
color: rgba(255, 255, 255, 0.4);
font-size: 15px;
font-weight: 400;
width: @fe-sidebar-width;
padding: 20px;
.text-overflow();
& > i {
margin-right: 10px;
}
}

View File

@@ -0,0 +1,101 @@
/*--------------------------
Base
----------------------------*/
@font-family-sans-serif : 'Lato', sans-serif;
@font-family-icon : 'fontAwesome';
@body-bg : #edecec;
@text-color : #8e8e8e;
@font-size-base : 15px;
@link-color : #46a5e0;
@link-hover-decoration : none;
/*--------------------------
File Explorer
----------------------------*/
@fe-sidebar-width : 320px;
@text-muted-color : #BDBDBD;
@text-strong-color : #333;
/*--------------------------
Colors
----------------------------*/
@cyan : #2ED2FF;
@amber : #ffc107;
@red : #ff726f;
@grey : #f5f5f5;
@dark-blue : #0084d3;
@blue : #00a6f7;
@white : #ffffff;
@black : #1b1e25;
@blue : #50b2ff;
@light-blue : #c1d1e8;
@green : #33d46f;
@yellow : #FFC107;
@orange : #ffc155;
@purple : #9C27B0;
@teal : #009688;
@brown : #795548;
@blue-gray : #374952;
@dark-gray : #32393F;
/*--------------------------
Dropdown
----------------------------*/
@dropdown-fallback-border : transparent;
@dropdown-border : transparent;
@dropdown-divider-bg : '';
@dropdown-link-hover-bg : rgba(0,0,0,0.05);
@dropdown-link-color : @text-color;
@dropdown-link-hover-color : #333;
@dropdown-link-disabled-color : #e4e4e4;
@dropdown-divider-bg : rgba(0,0,0,0.08);
@dropdown-link-active-color : #333;
@dropdown-link-active-bg : rgba(0, 0, 0, 0.075);
@dropdown-shadow : 0 2px 10px rgba(0, 0, 0, 0.2);
/*--------------------------
Modal
----------------------------*/
@modal-content-fallback-border-color: transparent;
@modal-content-border-color: transparent;
@modal-backdrop-bg: rgba(0,0,0,0.1);
@modal-header-border-color: transparent;
@modal-title-line-height: transparent;
@modal-footer-border-color: transparent;
@modal-inner-padding: 30px 35px;
@modal-title-padding: 30px 35px 0px;
@modal-sm: 400px;
/*-------------------------
Buttons
--------------------------*/
@btn-border-radius-large: 2px;
@btn-border-radius-small: 2px;
@btn-border-radius-base: 2px;
/*-------------------------
Colors
--------------------------*/
@brand-primary: #2298f7;
@brand-success: #4CAF50;
@brand-info: #00BCD4;
@brand-warning: #FF9800;
@brand-danger: #FF5722;
/*-------------------------
Form
--------------------------*/
@input-border: #eee;
/*-------------------------
List
--------------------------*/
@list-row-selected-bg: #fbf2bf;
@list-row-even-bg: #fafafa;

View File

@@ -0,0 +1,39 @@
/*----------------------------
Bootstrap
-----------------------------*/
@import "../../node_modules/bootstrap/less/scaffolding.less";
@import "../../node_modules/bootstrap/less/variables.less";
@import "../../node_modules/bootstrap/less/grid.less";
@import "../../node_modules/bootstrap/less/mixins.less";
@import "../../node_modules/bootstrap/less/normalize.less";
@import "../../node_modules/bootstrap/less/dropdowns.less";
@import "../../node_modules/bootstrap/less/modals.less";
@import "../../node_modules/bootstrap/less/tooltip.less";
@import "../../node_modules/bootstrap/less/responsive-utilities.less";
/*----------------------------
App
-----------------------------*/
@import 'inc/mixin';
@import 'inc/variables';
@import 'inc/base';
@import 'inc/animate/animate';
@import 'inc/generics';
@import 'inc/font';
@import 'inc/form';
@import 'inc/buttons';
@import 'inc/misc';
@import 'inc/login';
@import 'inc/header';
@import 'inc/sidebar';
@import 'inc/list';
@import 'inc/file-explorer';
@import 'inc/ie-warning';
/*----------------------------
Boostrap
-----------------------------*/
@import 'inc/dropdown';
@import 'inc/alert';
@import 'inc/modal';

126
browser/build.js Normal file
View File

@@ -0,0 +1,126 @@
/*
* Minio Cloud Storage (C) 2016 Minio, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
var moment = require('moment')
var async = require('async')
var exec = require('child_process').exec
var fs = require('fs')
var isProduction = process.env.NODE_ENV == 'production' ? true : false
var assetsFileName = ''
var commitId = ''
var date = moment.utc()
var version = date.format('YYYY-MM-DDTHH:mm:ss') + 'Z'
var releaseTag = date.format('YYYY-MM-DDTHH-mm-ss') + 'Z'
var buildType = 'DEVELOPMENT'
if (process.env.MINIO_UI_BUILD) buildType = process.env.MINIO_UI_BUILD
rmDir = function(dirPath) {
try { var files = fs.readdirSync(dirPath); }
catch(e) { return; }
if (files.length > 0)
for (var i = 0; i < files.length; i++) {
var filePath = dirPath + '/' + files[i];
if (fs.statSync(filePath).isFile())
fs.unlinkSync(filePath);
else
rmDir(filePath);
}
fs.rmdirSync(dirPath);
};
async.waterfall([
function(cb) {
rmDir('production');
rmDir('dev');
var cmd = 'webpack -p --config webpack.production.config.js'
if (!isProduction) {
cmd = 'webpack';
}
console.log('Running', cmd)
exec(cmd, cb)
},
function(stdout, stderr, cb) {
if (isProduction) {
fs.renameSync('production/index_bundle.js',
'production/index_bundle-' + releaseTag + '.js')
} else {
fs.renameSync('dev/index_bundle.js',
'dev/index_bundle-' + releaseTag + '.js')
}
var cmd = 'git log --format="%H" -n1'
console.log('Running', cmd)
exec(cmd, cb)
},
function(stdout, stderr, cb) {
if (!stdout) throw new Error('commitId is empty')
commitId = stdout.replace('\n', '')
if (commitId.length !== 40) throw new Error('commitId invalid : ' + commitId)
assetsFileName = 'ui-assets.go';
var cmd = 'go-bindata-assetfs -pkg browser -nocompress=true production/...'
if (!isProduction) {
cmd = 'go-bindata-assetfs -pkg browser -nocompress=true dev/...'
}
console.log('Running', cmd)
exec(cmd, cb)
},
function(stdout, stderr, cb) {
var cmd = 'gofmt -s -w -l bindata_assetfs.go'
console.log('Running', cmd)
exec(cmd, cb)
},
function(stdout, stderr, cb) {
fs.renameSync('bindata_assetfs.go', assetsFileName)
fs.appendFileSync(assetsFileName, '\n')
fs.appendFileSync(assetsFileName, 'var UIReleaseTag = "' + buildType + '.' +
releaseTag + '"\n')
fs.appendFileSync(assetsFileName, 'var UICommitID = "' + commitId + '"\n')
fs.appendFileSync(assetsFileName, 'var UIVersion = "' + version + '"')
fs.appendFileSync(assetsFileName, '\n')
var contents;
if (isProduction) {
contents = fs.readFileSync(assetsFileName, 'utf8')
.replace(/_productionIndexHtml/g, '_productionIndexHTML')
.replace(/productionIndexHtmlBytes/g, 'productionIndexHTMLBytes')
.replace(/productionIndexHtml/g, 'productionIndexHTML')
.replace(/_productionIndex_bundleJs/g, '_productionIndexBundleJs')
.replace(/productionIndex_bundleJsBytes/g, 'productionIndexBundleJsBytes')
.replace(/productionIndex_bundleJs/g, 'productionIndexBundleJs')
.replace(/_productionJqueryUiMinJs/g, '_productionJqueryUIMinJs')
.replace(/productionJqueryUiMinJsBytes/g, 'productionJqueryUIMinJsBytes')
.replace(/productionJqueryUiMinJs/g, 'productionJqueryUIMinJs');
} else {
contents = fs.readFileSync(assetsFileName, 'utf8')
.replace(/_devIndexHtml/g, '_devIndexHTML')
.replace(/devIndexHtmlBytes/g, 'devIndexHTMLBytes')
.replace(/devIndexHtml/g, 'devIndexHTML')
.replace(/_devIndex_bundleJs/g, '_devIndexBundleJs')
.replace(/devIndex_bundleJsBytes/g, 'devIndexBundleJsBytes')
.replace(/devIndex_bundleJs/g, 'devIndexBundleJs')
.replace(/_devJqueryUiMinJs/g, '_devJqueryUIMinJs')
.replace(/devJqueryUiMinJsBytes/g, 'devJqueryUIMinJsBytes')
.replace(/devJqueryUiMinJs/g, 'devJqueryUIMinJs');
}
contents = contents.replace(/MINIO_UI_VERSION/g, version)
contents = contents.replace(/index_bundle.js/g, 'index_bundle-' + releaseTag + '.js')
fs.writeFileSync(assetsFileName, contents, 'utf8')
console.log('UI assets file :', assetsFileName)
cb()
}
], function(err) {
if (err) return console.log(err)
})

Some files were not shown because too many files have changed in this diff Show More